aclnnInstanceNorm

Applicable Products

Function

• Description: Performs the Instance Normalization operation. Compared with aclnnBatchNorm, aclnnInstanceNorm normalizes each sample instance rather than the entire batch, making it more suitable for processing data such as images.

• Formula:

  $$y = {{x-E(x)}\over\sqrt {Var(x)+eps}} * gamma + beta$$

Function Prototype

Each operator uses a two-phase API. You must first call the aclnnInstanceNormGetWorkspaceSize API to obtain the input parameters and calculate the required workspace size based on the computation process, and then call the aclnnInstanceNorm API to perform the computation.

aclnnStatus aclnnInstanceNormGetWorkspaceSize(
  const aclTensor *x,
  const aclTensor *gamma,
  const aclTensor *beta,
  const char      *dataFormat,
  double           eps,
  aclTensor       *y,
  aclTensor       *mean,
  aclTensor       *variance,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)

aclnnStatus aclnnInstanceNorm(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnInstanceNormGetWorkspaceSize

• Parameters

  Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
  x (aclTensor*)	Input	Indicates the input data for the InstanceNorm computation, corresponding to `x` in the formula.	Supports null tensor. The actual data format is determined by the parameter `dataFormat`.	FLOAT32, FLOAT16	ND	4	√
  gamma (aclTensor*)	Input	Indicates the scaling factor (weight) for InstanceNorm computation, corresponding to `gamma` in the formula.	Supports null tensor. The data type remains consistent with that of x. The shape aligns with the C-axis of the input x.	FLOAT32, FLOAT16	ND	1	√
  beta (aclTensor*)	Input	Indicates the bias for InstanceNorm computation, corresponding to `beta` in the formula.	Supports null tensor. The data type must be consistent with that of `x`. The shape must align with the C axis of the input `x`.	FLOAT32, FLOAT16	ND	1	√
  dataFormat (char)	Input	Indicates the actual data format of the operator input tensor.	Supports NHWC or NCHW.	-	-	-	-
  eps (double)	Input	Indicates the value added to the variance to avoid division by zero. Corresponds to `eps` in the formula.	-	-	-	-	-
  y (aclTensor*)	Output	Indicates the output result of InstanceNorm, corresponding to `y` in the formula.	Supports null tensor. The shape and data type remain consistent with the shape and data type of the input `x`.	FLOAT32, FLOAT16	ND	4	√
  mean (aclTensor*)	Output	Indicates the mean of InstanceNorm, corresponding to `E(x)` in the formula.	Supports null tensor. The data type must be consistent with that of the input `x`. The shape satisfies the broadcast relationship with the input `x` (the shape of the first two dimensions is the same as that of the first two dimensions of the input `x`; the first two dimensions indicate the dimensions that do not require normalization, and the remaining dimensions have a size of 1).	FLOAT32, FLOAT16	ND	4	√
  variance (aclTensor*)	Output	Indicates the variance of InstanceNorm, corresponding to `Var(x)` in the formula.	Supports null tensor. The data type is consistent with the data type of `x`. The shape satisfies the broadcast relationship with the input `x` (the first two dimensions have the same shape as the first two dimensions of the input `x`, where the first two dimensions indicate the dimensions that do not require normalization, and the remaining dimensions have a size of 1).	FLOAT32, FLOAT16, BFLOAT16	ND	4	√
  workspaceSize (uint64_t*)	Output	Returns the workspace size that the user needs to allocate on the Device side.	-	-	-	-	-
  executor (aclOpExecutor**)	Output	Returns the operator executor, which contains the operator computation process.	-	-	-	-	-

• Return Value

  aclnnStatus: return code. For details, see aclnn Return Code.

  The first-phase API performs input parameter validation and returns an error in the following scenarios:

  Return Code	Error Code	Description
  ACLNN_ERR_PARAM_NULLPTR	161001	The input x, gamma, beta, y, mean, or variance is a null pointer.
  ACLNN_ERR_PARAM_INVALID	161002	The data types of the input x, gamma, beta, y, mean, and variance are not within the supported range.
  		The dimension of x is not 4, or the dimension of gamma/beta is not 1.
  		The shapes of gamma and beta are not aligned with the C axis of x.
  		Unsupported product model.
  		dataFormat is not set to NCHW or NHWC.
  ACLNN_ERR_INNER_NULLPTR	561103	Nullptr occurs in the intermediate computation result of the aclnn API.
  		The length of the C axis or H*W of x and y is less than 32 bytes.
  ACLNN_ERR_INNER_CREATE_EXECUTOR	561101	Failed to create aclOpExecutor inside the API.

aclnnInstanceNorm

• Parameters

  Parameter	Input/Output	Description
  workspace	Input	Address of the workspace memory allocated on the Device side.
  workspaceSize	Input	Size of the workspace allocated on the Device side, obtained through the first-phase API aclnnInstanceNormGetWorkspaceSize.
  executor	Input	Operator executor, which contains the operator computation flow.
  stream	Input	Specifies the stream for task execution.

• Return Value

  aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

• Functional dimensions:

• Data type support:

• x, gamma, beta, y, mean, and variance support: FLOAT32 and FLOAT16.

  • Data format support: ND.

• x and y require 4-dimensional shapes. gamma and beta require 1-dimensional shapes and must align with the C-axis of x and y.

• The H*W size of x and y must be greater than or equal to 32 bytes, and the C-axis must be greater than or equal to 32 bytes.

• The dataFormat parameter supports only "NHWC" and "NCHW".

• Boundary value scenario description:

  • When the input is Inf, the output is Inf.

  • When the input is NaN, the output is NaN.

• Deterministic computation:

  • aclnnInstanceNorm defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_instance_norm.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shapeSize = 1;
    for (auto i : shape) {
        shapeSize *= i;
    }
    return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // Boilerplate: resource initialization.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
int CreateAclTensor(
    const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
    aclTensor** tensor)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    // Call aclrtMalloc to allocate device-side memory.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
    // Copy the host-side data to the device-side memory by calling aclrtMemcpy.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

    // Calculate the strides of the contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    // Create an aclTensor by calling aclCreateTensor.
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
    return 0;
}

int main()
{
    // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
    // Set deviceId based on the actual device.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    // Handle according to your own needs.
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

    int64_t N = 1;
    int64_t C = 8;
    int64_t H = 4;
    int64_t W = 4;

    // 2. Construct inputs and outputs. Custom construction is required based on the API interface. This example calls the case without the optional bias input and the case with the bias input once each.
    std::vector<int64_t> xShape = {N, C, H, W};
    std::vector<int64_t> weightShape = {C};
    std::vector<int64_t> yShape = {N, C, H, W};
    std::vector<int64_t> reduceShape = {N, C, 1, 1};

    void* xDeviceAddr = nullptr;
    void* gammaDeviceAddr = nullptr;
    void* betaDeviceAddr = nullptr;
    void* yDeviceAddr = nullptr;
    void* meanDeviceAddr = nullptr;
    void* varianceDeviceAddr = nullptr;

    aclTensor* x = nullptr;
    aclTensor* gamma = nullptr;
    aclTensor* beta = nullptr;
    aclTensor* y = nullptr;
    aclTensor* mean = nullptr;
    aclTensor* variance = nullptr;

    std::vector<float> xHostData(N * C * H * W, 0.77);
    std::vector<float> gammaHostData(C, 1.5);
    std::vector<float> betaHostData(C, 0.5);
    std::vector<float> yHostData(N * C * H * W, 0.0);
    std::vector<float> meanHostData(N * C, 0.0);
    std::vector<float> varianceHostData(N * C, 0.0);
    const char* dataFormat = "NCHW";
    double eps = 1e-5;

    // Create the self aclTensor.
    ret = CreateAclTensor(xHostData, xShape, &xDeviceAddr, aclDataType::ACL_FLOAT, &x);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensor(gammaHostData, weightShape, &gammaDeviceAddr, aclDataType::ACL_FLOAT, &gamma);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensor(betaHostData, weightShape, &betaDeviceAddr, aclDataType::ACL_FLOAT, &beta);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensor(yHostData, yShape, &yDeviceAddr, aclDataType::ACL_FLOAT, &y);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensor(meanHostData, reduceShape, &meanDeviceAddr, aclDataType::ACL_FLOAT, &mean);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    ret = CreateAclTensor(varianceHostData, reduceShape, &varianceDeviceAddr, aclDataType::ACL_FLOAT, &variance);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    // Example of aclnnInstanceNorm API call.
    // Call the first-phase API of aclnnInstanceNorm.
    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;
    LOG_PRINT("\nUse aclnnInstanceNorm Non-Bias Port.");
    ret = aclnnInstanceNormGetWorkspaceSize(
        x, gamma, beta, dataFormat, eps, y, mean, variance, &workspaceSize, &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnInstanceNormGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);

    // Allocate device memory based on the workspaceSize calculated by the first-phase API.
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret;);
    }

    // Call the second-phase API of aclnnInstanceNorm.
    ret = aclnnInstanceNorm(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnInstanceNorm failed. ERROR: %d\n", ret); return ret);

    // 4. (Boilerplate) Synchronize and wait for task execution to complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

    // 5. Obtain the output values and copy the results from the device side to the host side. Modify this based on the specific API definition.

    // 5.1 Copy the output without bias.
    auto size = GetShapeSize(yShape);
    std::vector<float> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), yDeviceAddr, size * sizeof(resultData[0]),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    LOG_PRINT("==== InstanceNorm non-bias: y output");
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
    }

    auto outputMeanSize = GetShapeSize(reduceShape);
    std::vector<float> resultDataMean(outputMeanSize, 0);
    ret = aclrtMemcpy(
        resultDataMean.data(), resultDataMean.size() * sizeof(resultDataMean[0]), meanDeviceAddr,
        outputMeanSize * sizeof(resultDataMean[0]), ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    LOG_PRINT("==== InstanceNorm non-bias: mean output");
    for (int64_t i = 0; i < outputMeanSize; i++) {
        LOG_PRINT("result[%ld] is: %f\n", i, resultDataMean[i]);
    }

    auto outputVarSize = GetShapeSize(reduceShape);
    std::vector<float> resultDataVar(outputVarSize, 0);
    ret = aclrtMemcpy(
        resultDataVar.data(), resultDataVar.size() * sizeof(resultDataVar[0]), varianceDeviceAddr,
        outputVarSize * sizeof(resultDataVar[0]), ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    LOG_PRINT("==== InstanceNorm non-bias: rstd output");
    for (int64_t i = 0; i < outputVarSize; i++) {
        LOG_PRINT("result[%ld] is: %f\n", i, resultDataVar[i]);
    }

    // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
    aclDestroyTensor(x);
    aclDestroyTensor(gamma);
    aclDestroyTensor(beta);
    aclDestroyTensor(y);
    aclDestroyTensor(mean);
    aclDestroyTensor(variance);

    // 7. Release device resources. Modify based on the specific API definition.
    aclrtFree(xDeviceAddr);
    aclrtFree(gammaDeviceAddr);
    aclrtFree(betaDeviceAddr);

    aclrtFree(yDeviceAddr);
    aclrtFree(meanDeviceAddr);
    aclrtFree(varianceDeviceAddr);

    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }

    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}