aclnnLayerNorm&aclnnLayerNormWithImplMode

Supported Products

Product	Supported
Atlas A3 training series products/Atlas A3 inference series products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference series products	×
Atlas training series products	√

Function

Description: Performs normalization with a mean of 0 and a standard deviation of 1 on a specified layer. Compared with aclnnLayerNorm, aclnnLayerNormWithImplMode allows you to select different normalization policies by configuring the impl_mode parameter to adapt to different application scenarios and performance requirements.
Formula:
$out = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + eps}} * weightOptional + biasOptional$
E[x] indicates the input's mean value, and Var[x] indicates the input's variance.

Prototype

Each operator has two-phase API calls. First, aclnnLayerNormGetWorkspaceSize or aclnnLayerNormWithImplModeGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnLayerNorm or aclnnLayerNormWithImplMode is called to perform computation.

aclnnStatus aclnnLayerNormGetWorkspaceSize(
  const aclTensor   *input,
  const aclIntArray *normalizedShape,
  const aclTensor   *weightOptional,
  const aclTensor   *biasOptional,
  double             eps,
  aclTensor         *out,
  aclTensor         *meanOutOptional,
  aclTensor         *rstdOutOptional,
  uint64_t          *workspaceSize,
  aclOpExecutor    **executor)

aclnnStatus aclnnLayerNorm(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnStatus aclnnLayerNormWithImplModeGetWorkspaceSize(
  const aclTensor    *input,
  const aclIntArray  *normalizedShape,
  const aclTensor    *weightOptional,
  const aclTensor    *biasOptional,
  double              eps,
  aclTensor          *out,
  aclTensor          *meanOutOptional,
  aclTensor          *rstdOutOptional,
  int32_t             implMode,
  uint64_t           *workspaceSize,
  aclOpExecutor     **executor)

aclnnStatus aclnnLayerNormWithImplMode(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnLayerNormGetWorkspaceSize

Parameters:

Name	Input/Output	Description	Usage Notes	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
input	Input	Input for normalization computation, corresponding to `x` in the formula.	Empty tensors are supported. The shape is [A1,...,Ai,R1,...,Rj], where A1 to Ai indicate dimensions that do not require normalization, and R1 to Rj indicate dimensions that require normalization.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
normalizedShape	Input	Dimensions for normalization computation.	The value is [R1,...,Rj]. The length is less than or equal to the length of the input shape. The value cannot be empty.	INT64	-	-	-
weightOptional	Input	(Optional) Weight for normalization computation. It corresponds to `weightOptional` in the formula.	Empty tensors are supported. When `weightOptional` is not empty: The data type is the same as that of `input` or is FLOAT. When `biasOptional` exists, the data types of `weightOptional` and `biasOptional` are the same. The shape is the same as that of `normalizedShape`, that is, [R1,...,Rj]. When `weightOptional` is empty, the API constructs a tensor with shape [R1,...,Rj] and all data being 1. When `biasOptional` exists, the data types of `weightOptional` and `biasOptional` are the same. When `biasOptional` does not exist, the data types of `weightOptional` and `input` are the same.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
biasOptional	Input	(Optional) Offset for normalization computation. It corresponds to `biasOptional` in the formula.	Empty tensors are supported. When `biasOptional` is not empty: The data type is the same as that of `input` or is FLOAT. When `weightOptional` exists, the data types of `biasOptional` and `weightOptional` are the same. The shape is the same as that of `normalizedShape`, that is, [R1,...,Rj]. When `biasOptional` is empty, the API constructs a tensor with shape [R1,...,Rj] and all data being 0. When `weightOptional` exists, the data types of `biasOptional` and `weightOptional` are the same. When `weightOptional` does not exist, the data types of `biasOptional` and `input` are the same.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
eps	Input	Value added to the denominator to ensure numerical stability. It corresponds to `eps` in the formula.	-	DOUBLE	-	-	-
out	Output	Normalization result. It corresponds to `out` in the formula.	Empty tensors are supported. The data type is the same as that of `input`. The shape must be the same as that of `input`, that is, [A1,...,Ai,R1,...,Rj].	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
meanOutOptional	Output	(Optional) Normalized mean value. It corresponds to `E(x)` in the formula.	Empty tensors are supported. The data type is the same as that of `input`. When `rstdOutOptional` exists, the shape is the same as that of `rstdOutOptional`. The shape is [A1,...,Ai,1,...,1] with j 1s after Ai, which is the same as the length of the axis to be normalized.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
rstdOutOptional	Output	(Optional) Reciprocal of the normalized standard deviation. It corresponds to `Var(x)` in the formula.	Empty tensors are supported. The data type is the same as that of input. When `meanOutOptional` exists, the shape is the same as that of `meanOutOptional`. The shape is [A1,...,Ai,1,...,1] with j 1s after Ai, which is the same as the length of the axis to be normalized.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
workspaceSize	Output	Size of the workspace to be allocated on the device.	-	-	-	-	-
executor	Output	Operator executor, containing the operator computation process.	-	-	-	-	-

Atlas training series products and Atlas 200I/500 A2 inference products: The data type of input, weightOptional, biasOptional, out, meanOutOptional, and rstdOutOptional does not support BFLOAT16.

Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

The first-phase API implements input parameter verification. The following errors may be thrown:

Return	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The passed input, normalizedShape, or out is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The shape of input, normalizedShape, weightOptional (non-empty), biasOptional (non-empty), out, meanOutOptional (non-empty), or rstdOutOptional (non-empty) has more than eight dimensions.
		The data type of input, weightOptional (non-empty), biasOptional (non-empty), out, meanOutOptional (non-empty), or rstdOutOptional (non-empty) is not supported.
		normalizedShape has less than one dimension.
		weightOptional is specified and the shape is different from normalizedShape.
		biasOptional is specified and the shape is different from normalizedShape.
		The number of dimensions of input is less than that of normalizedShape.
		The shape of input is different from the shape of the corresponding dimension when right aligned with normalizedShape.
		The shapes of input and out are different.

aclnnLayerNorm

Parameters:

Name	Input/Output	Description
workspace	Input	Address of the workspace to be allocated on the device.
workspaceSize	Input	Size of the workspace to be allocated on the device, which is obtained by the first-phase API aclnnLayerNormGetWorkspaceSize.
executor	Input	Operator executor, containing the operator computation process.
stream	Input	Stream for executing the task.

Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

aclnnLayerNormWithImplModeGetWorkspaceSize

Parameters:

Name	Input/Output	Description	Usage Notes	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
input	Input	Input for normalization computation, corresponding to x in the formula.	Empty tensors are supported. The shape is [A1,...,Ai,R1,...,Rj], where A1 to Ai indicate dimensions that do not require normalization, and R1 to Rj indicate dimensions that require normalization.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
normalizedShape	Input	Dimensions for normalization computation.	The value is [R1,...,Rj]. The length is less than or equal to the length of the input shape. The value cannot be empty.	INT64	-	-	-
weightOptional	Input	(Optional) Weight for normalization computation. It corresponds to `weightOptional` in the formula.	Empty tensors are supported. When `weightOptional` is not empty: The data type is the same as that of `input` or is FLOAT32. When `biasOptional` exists, the data types of `weightOptional` and `biasOptional` are the same. The shape is the same as that of normalizedShape, that is, [R1,...,Rj]. When `weightOptional` is empty, the API constructs a tensor with shape [R1,...,Rj] and all data being 1. When biasOptional exists, the data types of weightOptional and biasOptional are the same. When biasOptional does not exist, the data types of weightOptional and input are the same.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
biasOptional	Input	(Optional) Offset for normalization computation. It corresponds to biasOptional in the formula.	Empty tensors are supported. When `biasOptional` is not empty: The data type is the same as that of `input` or is FLOAT32. When `weightOptional` exists, the data types of `biasOptional` and `weightOptional` are the same. The shape is the same as that of normalizedShape, that is, [R1,...,Rj]. When `biasOptional` is empty, the API constructs a tensor with shape [R1,...,Rj] and all data being 0. When weightOptional exists, the data types of biasOptional and weightOptional are the same. When weightOptional does not exist, the data types of biasOptional and input are the same.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
eps	Input	Value added to the denominator to ensure numerical stability. It corresponds to `eps` in the formula.	-	DOUBLE	-	-	-
out	Output	Normalization result. It corresponds to out in the formula.	Empty tensors are supported. The data type is the same as that of input. The shape must be the same as that of `input`, that is, [A1,...,Ai,R1,...,Rj].	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
meanOutOptional	Output	(Optional) Normalized mean value. It corresponds to E(x) in the formula.	Empty tensors are supported. The data type is the same as that of input. The shape is [A1,...,Ai,1,...,1] with j 1s after Ai, which is the same as the length of the axis to be normalized.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
rstdOutOptional	Output	(Optional) Reciprocal of the normalized standard deviation. It corresponds to Var(x) in the formula.	Empty tensors are supported. The data type is the same as that of input. The shape is [A1,...,Ai,1,...,1] with j 1s after Ai, which is the same as the length of the axis to be normalized.	FLOAT32, FLOAT16, BFLOAT16	ND	1-8	√
implMode	Input	Computation mode selected by the kernel.	The value can be 0, 1, or 2. The default value is 0. The value 0 indicates the high-precision mode, 1 indicates the high-performance mode, and 2 indicates the FLOAT16 computation mode. Exercise caution when deciding to use the high-performance mode, which causes precision drop. The FLOAT16 computation mode only supports inputs of FLOAT16, and the computation precision is the lowest.	INT32	-	-	-
workspaceSize	Output	Size of the workspace to be allocated on the device.	-	-	-	-	-
executor	Output	Operator executor, containing the operator computation process.	-	-	-	-	-

Atlas training series products and Atlas 200I/500 A2 inference products: The data type of <idp:inline displayname="code" id="code1978063914229">input, <idp:inline displayname="code" id="code8780123992215">weightOptional, <idp:inline displayname="code" id="code1878018398228">biasOptional, <idp:inline displayname="code" id="code177809396224">out, <idp:inline displayname="code" id="code197800392223">meanOutOptional, and <idp:inline displayname="code" id="code9780133912227">rstdOutOptional does not support BFLOAT16.

Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

The first-phase API implements input parameter verification. The following errors may be thrown:

Return	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The passed input, normalizedShape, or out is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The shape of input, normalizedShape, weightOptional (non-empty), biasOptional (non-empty), out, meanOutOptional (non-empty), or rstdOutOptional (non-empty) has more than eight dimensions.
		The data type of input, weightOptional (non-empty), biasOptional (non-empty), out, meanOutOptional (non-empty), or rstdOutOptional (non-empty) is not supported.
		normalizedShape has less than one dimension.
		weightOptional is specified and the shape is different from normalizedShape.
		biasOptional is specified and the shape is different from normalizedShape.
		The number of dimensions of input is less than that of normalizedShape.
		The shape of input is different from the shape of the corresponding dimension when right aligned with normalizedShape.
		The value of implMode is not 0, 1, or 2.
		The value of implMode is 2 and not all inputs are of the FLOAT16 type.
		The shapes of input and out are different.

aclnnLayerNormWithImplMode

Parameters:

Name	Input/Output	Description
workspace	Input	Address of the workspace to be allocated on the device.
workspaceSize	Input	Size of the workspace to be allocated on the device, which is obtained by the first-phase API aclnnLayerNormWithImplModeGetWorkspaceSize.
executor	Input	Operator executor, containing the operator computation process.
stream	Input	Stream for executing the task.

Returns:

aclnnStatus: status code. For details, see aclnn Return Codes.

Constraints

The shape of input, normalizedShape, weightOptional (non-empty), biasOptional (non-empty), out, meanOutOptional (non-empty), or rstdOutOptional (non-empty) cannot exceed eight dimensions.
Deterministic compute:
- aclnnLayerNorm defaults to a deterministic implementation.
- aclnnLayerNormWithImplMode defaults to a deterministic implementation.

Example

The following example is for reference only. For details, see Compilation and Running Sample.

Example for aclnnLayerNorm:

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_layer_norm.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shapeSize = 1;
    for (auto i : shape) {
        shapeSize *= i;
    }
    return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // (Fixed writing) Initialize resources.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
int CreateAclTensorMem(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
void aclCreateTensorP(const std::vector<T>& shape, void** deviceAddr, aclDataType dataType, aclTensor** tensor)
{
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
}

template <typename T>
void aclCreateIntArrayP(const std::vector<T>& hostData, aclIntArray** intArray)
{
    *intArray = aclCreateIntArray(hostData.data(), hostData.size());
}

int main()
{
    // 1. (Fixed writing) Initialize the device and stream. For details, see the ACL API manual.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

    // 2. Construct the input and output based on the API.
    std::vector<int64_t> xShape = {1, 2, 32};
    std::vector<int64_t> normShape = {32};
    std::vector<int64_t> meanShape = {1, 2, 1};
    void* xDeviceAddr = nullptr;
    void* weightDeviceAddr = nullptr;
    void* biasDeviceAddr = nullptr;
    void* outDeviceAddr = nullptr;
    void* meanDeviceAddr = nullptr;
    void* rstdDeviceAddr = nullptr;
    aclTensor* x = nullptr;
    aclIntArray* norm = nullptr;
    aclTensor* weight = nullptr;
    aclTensor* bias = nullptr;
    aclTensor* out = nullptr;
    aclTensor* mean = nullptr;
    aclTensor* rstd = nullptr;
    std::vector<float> xHostData(64, 2.0);
    std::vector<int64_t> normData = {32};
    std::vector<float> weightHostData(32, 1.0);
    std::vector<float> biasHostData(32, 0.0);
    std::vector<float> outHostData(64, 0.0);
    std::vector<float> meanHostData(2, 0.0);
    std::vector<float> rstdHostData(2, 0.0);
    double eps = 1e-5;

    ret = CreateAclTensorMem(xHostData, xShape, &xDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensorMem(weightHostData, normShape, &weightDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensorMem(biasHostData, normShape, &biasDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensorMem(outHostData, xShape, &outDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensorMem(meanHostData, meanShape, &meanDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensorMem(rstdHostData, meanShape, &rstdDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    aclCreateTensorP(xShape, &xDeviceAddr, aclDataType::ACL_FLOAT, &x);
    aclCreateIntArrayP(normData, &norm);
    aclCreateTensorP(normShape, &weightDeviceAddr, aclDataType::ACL_FLOAT, &weight);
    aclCreateTensorP(normShape, &biasDeviceAddr, aclDataType::ACL_FLOAT, &bias);
    aclCreateTensorP(xShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);
    aclCreateTensorP(meanShape, &meanDeviceAddr, aclDataType::ACL_FLOAT, &mean);
    aclCreateTensorP(meanShape, &rstdDeviceAddr, aclDataType::ACL_FLOAT, &rstd);

    // 3. Call the CANN operator library API, which needs to be replaced with the actual API.
    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;
    // Call the first-phase API of aclnnLayerNorm.
    ret = aclnnLayerNormGetWorkspaceSize(x, norm, weight, bias, eps, out, mean, rstd, &workspaceSize, &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLayerNormGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    }
    // Call the second-phase API of aclnnLayerNorm.
    ret = aclnnLayerNorm(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLayerNorm failed. ERROR: %d\n", ret); return ret);

    // 4. (Fixed writing) Wait until the task execution is complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

    // 5. Obtain the output value and copy the result from the device memory to the host. Modify the configuration based on the API definition.
    auto size = GetShapeSize(xShape);
    std::vector<float> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr, size * sizeof(resultData[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy first result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("out result[%ld] is: %f\n", i, resultData[i]);
    }

    auto size1 = GetShapeSize(meanShape);
    std::vector<float> resultData1(size1, 0);
    ret = aclrtMemcpy(
        resultData1.data(), resultData1.size() * sizeof(resultData1[0]), meanDeviceAddr, size1 * sizeof(resultData1[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy second result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size1; i++) {
        LOG_PRINT("mean result[%ld] is: %f\n", i, resultData1[i]);
    }

    auto size2 = GetShapeSize(meanShape);
    std::vector<float> resultData2(size2, 0);
    ret = aclrtMemcpy(
        resultData2.data(), resultData2.size() * sizeof(resultData2[0]), rstdDeviceAddr, size2 * sizeof(resultData2[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy last result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size2; i++) {
        LOG_PRINT("rstd result[%ld] is: %f\n", i, resultData2[i]);
    }

    // 6. Release aclTensor and aclIntArray. Modify the configuration based on the API definition.
    aclDestroyTensor(x);
    aclDestroyIntArray(norm);
    aclDestroyTensor(weight);
    aclDestroyTensor(bias);
    aclDestroyTensor(out);
    aclDestroyTensor(mean);
    aclDestroyTensor(rstd);

    // 7. Release device resources.
    aclrtFree(xDeviceAddr);
    aclrtFree(weightDeviceAddr);
    aclrtFree(biasDeviceAddr);
    aclrtFree(outDeviceAddr);
    aclrtFree(meanDeviceAddr);
    aclrtFree(rstdDeviceAddr);
    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}

Example for aclnnLayerNormWithImplMode:

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_layer_norm.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shapeSize = 1;
    for (auto i : shape) {
        shapeSize *= i;
    }
    return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // (Fixed writing) Initialize resources.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
int CreateAclTensorMem(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    // Call aclrtMalloc to allocate memory on the device.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
    // Call aclrtMemcpy to copy the data on the host to the memory on the device.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
void aclCreateTensorP(const std::vector<T>& shape, void** deviceAddr, aclDataType dataType, aclTensor** tensor)
{
    // Compute the strides of the contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    // Call aclCreateTensor to create an aclTensor.
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
}

template <typename T>
void aclCreateIntArrayP(const std::vector<T>& hostData, aclIntArray** intArray)
{
    // Call the API to create an aclIntArray.
    *intArray = aclCreateIntArray(hostData.data(), hostData.size());
}

int main()
{
    // 1. (Fixed writing) Initialize the device and stream. For details, see the ACL API manual.
    // Set the device ID in use.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

    // 2. Construct the input and output based on the API.
    std::vector<int64_t> xShape = {1, 2, 32};
    std::vector<int64_t> normShape = {32};
    std::vector<int64_t> meanShape = {1, 2, 1};
    void* xDeviceAddr = nullptr;
    void* weightDeviceAddr = nullptr;
    void* biasDeviceAddr = nullptr;
    void* outDeviceAddr = nullptr;
    void* meanDeviceAddr = nullptr;
    void* rstdDeviceAddr = nullptr;
    aclTensor* x = nullptr;
    aclIntArray* norm = nullptr;
    aclTensor* weight = nullptr;
    aclTensor* bias = nullptr;
    aclTensor* out = nullptr;
    aclTensor* mean = nullptr;
    aclTensor* rstd = nullptr;
    std::vector<float> xHostData(64, 2.0);
    std::vector<int64_t> normData = {32};
    std::vector<float> weightHostData(32, 1.0);
    std::vector<float> biasHostData(32, 0.0);
    std::vector<float> outHostData(64, 0.0);
    std::vector<float> meanHostData(2, 0.0);
    std::vector<float> rstdHostData(2, 0.0);
    double eps = 1e-5;
    int32_t implMode = 0;

    // Create an x aclTensor.
    ret = CreateAclTensorMem(xHostData, xShape, &xDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create a weight aclTensor.
    ret = CreateAclTensorMem(weightHostData, normShape, &weightDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create a bias aclTensor.
    ret = CreateAclTensorMem(biasHostData, normShape, &biasDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create an out aclTensor.
    ret = CreateAclTensorMem(outHostData, xShape, &outDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create a mean aclTensor.
    ret = CreateAclTensorMem(meanHostData, meanShape, &meanDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create a rstd aclTensor.
    ret = CreateAclTensorMem(rstdHostData, meanShape, &rstdDeviceAddr);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    aclCreateTensorP(xShape, &xDeviceAddr, aclDataType::ACL_FLOAT, &x);
    aclCreateIntArrayP(normData, &norm);
    aclCreateTensorP(normShape, &weightDeviceAddr, aclDataType::ACL_FLOAT, &weight);
    aclCreateTensorP(normShape, &biasDeviceAddr, aclDataType::ACL_FLOAT, &bias);
    aclCreateTensorP(xShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);
    aclCreateTensorP(meanShape, &meanDeviceAddr, aclDataType::ACL_FLOAT, &mean);
    aclCreateTensorP(meanShape, &rstdDeviceAddr, aclDataType::ACL_FLOAT, &rstd);

    // 3. Call the CANN operator library API, which needs to be replaced with the actual API.
    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;
    // Call the first-phase API of aclnnLayerNormWithImplMode.
    ret = aclnnLayerNormWithImplModeGetWorkspaceSize(
        x, norm, weight, bias, eps, out, mean, rstd, implMode, &workspaceSize, &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLayerNormWithImplModeGetWorkspaceSize failed. ERROR: %d\n", ret);
              return ret);
    // Allocate device memory based on workspaceSize computed by the first-phase API.
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    }
    // Call the second-phase API of aclnnLayerNormWithImplMode.
    ret = aclnnLayerNormWithImplMode(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnLayerNormWithImplMode failed. ERROR: %d\n", ret); return ret);

    // 4. (Fixed writing) Wait until the task execution is complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

    // 5. Obtain the output value and copy the result from the device memory to the host. Modify the configuration based on the API definition.
    auto size = GetShapeSize(xShape);
    std::vector<float> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr, size * sizeof(resultData[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy first result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("out result[%ld] is: %f\n", i, resultData[i]);
    }

    auto size1 = GetShapeSize(meanShape);
    std::vector<float> resultData1(size1, 0);
    ret = aclrtMemcpy(
        resultData1.data(), resultData1.size() * sizeof(resultData1[0]), meanDeviceAddr, size1 * sizeof(resultData1[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy second result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size1; i++) {
        LOG_PRINT("mean result[%ld] is: %f\n", i, resultData1[i]);
    }

    auto size2 = GetShapeSize(meanShape);
    std::vector<float> resultData2(size2, 0);
    ret = aclrtMemcpy(
        resultData2.data(), resultData2.size() * sizeof(resultData2[0]), rstdDeviceAddr, size2 * sizeof(resultData2[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy last result from device to host failed. ERROR: %d\n", ret);
              return ret);
    for (int64_t i = 0; i < size2; i++) {
        LOG_PRINT("rstd result[%ld] is: %f\n", i, resultData2[i]);
    }

    // 6. Release aclTensor and aclIntArray. Modify the configuration based on the API definition.
    aclDestroyTensor(x);
    aclDestroyIntArray(norm);
    aclDestroyTensor(weight);
    aclDestroyTensor(bias);
    aclDestroyTensor(out);
    aclDestroyTensor(mean);
    aclDestroyTensor(rstd);

    // 7. Release device resources.
    aclrtFree(xDeviceAddr);
    aclrtFree(weightDeviceAddr);
    aclrtFree(biasDeviceAddr);
    aclrtFree(outDeviceAddr);
    aclrtFree(meanDeviceAddr);
    aclrtFree(rstdDeviceAddr);
    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}