aclnnQuantConvolutionWeightNz

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	×
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	×
Atlas training products	×

Function

Description: Performs per-channel quantized 3D convolution computation. The weight supports only the FRACTAL_Z_3D format. The convolution computation process is consistent with the aclnnConvolution API.
Formula:

Assume the input shape is $(N, C_{\text{in}}, D, H, W)$ , the original weight shape is $(C_{\text{out}}, C_{\text{in}}, K_d, K_h, K_w)$ , the scale shape is $(C_{\text{out}})$ , the bias shape is $C_{\text{out}}$ , and the output shape is $(N, C_{\text{out}}, D_{\text{out}}, H_{\text{out}}, W_{\text{out}})$ , where $N$ denotes the batch size, $C$ is the number of channels, $D$ , $H$ , and $W$ are the depth, height, and width of the sample respectively, and $K_d$ , $K_h$ , and $K_w$ are the depth, height, and width of the convolution kernel respectively. The output is expressed as:

\text{output}(N_i, C_{\text{out}_j}, D_{\text{out}}, H_{\text{out}}, W_{\text{out}}) = \left[\sum_{k = 0}^{C_{\text{in}} - 1} \text{weight}(C_{\text{out}_j}, k) \star \text{input}(N_i, k)\right] \times \text{scale}(C_{\text{out}_j}) + \text{bias}(C_{\text{out}_j})

Where $\star$ denotes the convolution operation, which depends on the convolution input dimensions and the convolution type (dilated convolution, grouped convolution). $N$ represents the batch size, $C$ represents the number of channels, and $D$ , $H$ , and $W$ represent the depth, height, and width respectively. The formulas for the corresponding output dimensions are as follows:

D_{\text{out}}=[(D + 2 \times padding[0] - dilation[0] \times (K_d - 1) - 1 ) / stride[0]] + 1 \\ H_{\text{out}}=[(H + 2 \times padding[1] - dilation[1] \times (K_h - 1) - 1 ) / stride[1]] + 1 \\ W_{\text{out}}=[(W + 2 \times padding[2] - dilation[2] \times (K_w - 1) - 1 ) / stride[2]] + 1

Function Prototype

Each operator is divided into a two-phase API. You must first call the aclnnQuantConvolutionWeightNzGetWorkspaceSize API to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call the aclnnQuantConvolutionWeightNz API to perform the computation.

aclnnStatus aclnnQuantConvolutionWeightNzGetWorkspaceSize(
    const aclTensor       *input,
    const aclTensor       *weight,
    const aclTensor       *bias,
    const aclTensor       *scale,
    const aclTensor       *offset,
    const aclIntArray     *stride,
    const aclIntArray     *padding,
    const aclIntArray     *dilation,
    bool                   transposed,
    const aclIntArray     *outputPadding,
    int64_t                groups,
    int32_t                offsetx,
    const char            *roundMode,
    aclTensor             *output,
    uint64_t              *workspaceSize,
    aclOpExecutor         **executor)

aclnnStatus aclnnQuantConvolutionWeightNz(
    void            *workspace,
    const uint64_t   workspaceSize,
    aclOpExecutor   *executor,
    aclrtStream      stream)

aclnnQuantConvolutionWeightNzGetWorkspaceSize

Parameters

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
Input (aclTensor*)	Input	input in the formula, representing the convolution input.	The dimensions of input, the original shape of weight, and output must be the same. Empty tensors are not supported. The data types of input and weight must satisfy the data type deduction rules (see Deduction Relationship). N≥0, C≥1, D≥0, H≥0, W≥0.	INT8	NCDHW	5	√
weight (aclTensor*)	Input	weight in the formula, representing the convolution weight.	The C dimension of its original shape must be consistent with the C dimension of input. Empty tensors are not supported. The data type must satisfy the data type deduction rules with the data type of input (see Deduction Relationship). FRACTAL_Z_3D is a 3D convolution weight format. The dimensions of the format are: (D * C1 * H * W, N1, N0, C0), where D, H, and W are NCDWH before transdata. N0=16, C0=32, and N1 satisfies the relationship with N: N1=CEIL(N, N0). All dimensions ≥ 1.	INT8	Fractal_Z_3D	4	√
bias (aclTensor*)	Input	Represents the convolution bias in the formula.	One-dimensional and equal to the first dimension of the original shape of weight.	BFLOAT16, FLOAT16, FLOAT	ND	1	√
scale (aclTensor*)	Input	The scale in the formula, representing the quantization parameter.	One-dimensional and equal to the first dimension of the original shape of weight.	FLOAT	ND	1	√
offset (aclTensor*)	Input	Reserved quantization parameter.	Currently not supported. Pass a null pointer nullptr.	-	-	-	-
stride (aclIntArray*)	Input	Convolution stride.	Array length = 3. strideH and strideW must be in the range [1, 63]. In conv3d scenarios, strideD must be in the range [1, 1000000].	INT32	-	-	-
padding (aclIntArray*)	Input	Padding for the input.	The value must be ≥ 0. paddingH and paddingW must be in the range [0, 255]. In the conv3d scenario, paddingD must be in the range [0, 1000000].	INT32	-	-	-
dilation (aclIntArray*)	Input	Spacing between elements in the convolution kernel.	The value must be greater than 0. dilationH and dilationW must be in the range [1, 255]. In conv3d scenarios, dilationD must be in the range [1, 1000000].	INT32	-	-	-
transposed (bool)	Input	Reserved parameter. Indicates whether it is a transposed quantized convolution.	Currently not supported. Pass in false.	BOOL	-	-	-
outputPadding (aclIntArray*)	Input	Reserved parameter. Specifies the padding for all edges of the output in the case of transposed convolution.	Ignored for non-transposed convolution. Currently not supported. Pass a null pointer nullptr.	INT32	-	-	-
groups (int64_t)	Input	Number of block links from the input channel to the output channel.	Value = 1	INT64	-	-	-
offsetx (int32_t)	Input	Represents the quantization factor.	[-128, 127] or 0.	INT32	-	-	-
roundMode (char*)	Input	Indicates the rounding mode.	rint, round, or nullptr.	CHAR*	-	-	-
output（aclTensor*）	Output	output in the formula, representing the convolution output.	Its shape follows the convolution deduction rules. Empty tensor output is not supported. The number of channels equals the first dimension of the original weight shape, and other dimensions are ≥ 0.	BFLOAT16, FLOAT16	NCDHW	5	√
workspaceSize (uint64_t*)	Output	Returns the workspace size to be applied for on the Device side.	-	-	-	-	-
executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return Value

aclnnStatus: Return status code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Value	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	A pointer-type input parameter is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The data types and data formats of input, weight, bias, scale, offset, and output are not within the supported range.
		The input shapes of stride, padding, and dilation are incorrect.
		The groups input is incorrect.
		The input shapes of scale and bias are incorrect.
		The shape of output does not match the infershape result.
		Any dimension of the input tensor being zero does not meet the requirement.
		The spatial size of input after padding is smaller than the spatial size of weight (after dilation expansion if dilation > 1).
		The number of channels of weight and input does not meet the requirement.
		The shape of weight before and after conversion does not satisfy the conversion relationship.
		stride or dilation being less than 0 does not meet the requirement.
		The current processor does not support convolution.
ACLNN_ERR_INNER_NULLPTR	561103	An internal API validation error, typically caused by input data or attribute specifications falling outside the supported range.
ACLNN_ERR_RUNTIME_ERROR	361001	An exception occurs when the API calls an NPU runtime interface, for example, SocVersion is not supported.

aclnnQuantConvolutionWeightNz

Parameters

Parameter	Input/Output	Description
workspace	Input	Workspace memory address applied for on the Device side.
workspaceSize	Input	Workspace size applied for on the Device side, obtained from the first-phase API aclnnQuantConvolutionWeightNzGetWorkspaceSize.
executor	Input	Operator executor that contains the operator computation flow.
stream	Input	Specifies the stream for task execution.

Return Value

aclnnStatus: Return status code. For details, see aclnn Return Code.

Constraints

Deterministic computation
- aclnnQuantConvolutionWeightNz defaults to a deterministic implementation.

Constraint Type	Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products
Input, weight	-
Bias	-
Scale	-
padding	The array length of padding must be 3.
groups	The value of groups must be 1.
offsetx	offsetx is not supported. Pass 0.
roundMode	roundMode is not supported. Pass a null pointer nullptr.
output	-
Other Constraints	The operator can only be called in inference scenarios. Only forward 3D convolution is supported. For each tensor in the input, weight original shape, bias, and scale groups, the size of each dimension must be less than 1000000.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

Use different main functions based on the product model.

#include <iostream>
#include <memory>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_npu_format_cast.h"
#include "aclnnop/aclnn_quant_convolution.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define CHECK_FREE_RET(cond, return_expr) \
  do {                                    \
    if (!(cond)) {                        \
      Finalize(deviceId, stream);         \
      return_expr;                        \
    }                                     \
  } while (0)

#define LOG_PRINT(message, ...)      \
  do {                               \
    printf(message, ##__VA_ARGS__);  \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shape_size = 1;
  for (auto i: shape) {
    shape_size *= i;
  }
  return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for device memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Call aclrtMemcpy to copy host data to device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_NCDHW,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

template <typename T>
int CreateAclTensorWithFormat(const std::vector<T>& hostData, const std::vector<int64_t>& shape, int64_t** storageShape, uint64_t* storageShapeSize, void** deviceAddr,
                              aclDataType dataType, aclTensor** tensor, aclFormat format) {
    auto size = hostData.size() * sizeof(T);
    // Call aclrtMalloc to apply for device memory.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
    // Copy host-side data to device memory by calling aclrtMemcpy.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

    // Calculate the strides of a contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0,
                                format, *storageShape, *storageShapeSize, *deviceAddr);
    return 0;
}

template <typename T>
int CreateAclTensorND(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                      aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Allocate device memory by calling aclrtMalloc.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Copy host-side data to device memory by calling aclrtMemcpy.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

void Finalize(int32_t deviceId, aclrtStream& stream)
{
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
}

int aclnnQuantConvolutionWeightNzTest(int32_t deviceId, aclrtStream& stream, std::vector<aclDataType> dtypesInfo)
{
  auto ret = Init(deviceId, &stream);
  // Perform check as needed.
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API.
  std::vector<int64_t> shapeInput = {2, 2, 32, 32, 32};
  std::vector<int64_t> shapeWeight = {2, 2, 3, 3, 3};
  std::vector<int64_t> shapeScale = {2};
  std::vector<int64_t> shapeBias = {2};
  std::vector<int64_t> shapeResult = {2, 2, 32, 32, 32};
  std::vector<int64_t> convStrides;
  std::vector<int64_t> convPads;
  std::vector<int64_t> convOutPads;
  std::vector<int64_t> convDilations;

  void* deviceDataA = nullptr;
  void* deviceDataB = nullptr;
  void* deviceDataBNz = nullptr;
  void* deviceDataScale = nullptr;
  void* deviceDataBias = nullptr;
  void* deviceDataResult = nullptr;

  aclTensor* input = nullptr;
  aclTensor* weight = nullptr;
  aclTensor* weightNz = nullptr;
  aclTensor* scale= nullptr;
  aclTensor* bias= nullptr;
  aclTensor* result = nullptr;
  std::vector<int8_t> inputData(GetShapeSize(shapeInput), 1);
  std::vector<int8_t> weightData(GetShapeSize(shapeWeight), 1);
  std::vector<float> biasData(GetShapeSize(shapeBias), 1);
  std::vector<float> scaleData(GetShapeSize(shapeScale), 1);
  std::vector<uint16_t> outputData(GetShapeSize(shapeResult), 1);
  convStrides = {1, 1, 1};
  convPads = {1, 1, 1};
  convOutPads = {1, 1, 1};
  convDilations = {1, 1, 1};
  aclDataType inputDtype = dtypesInfo[0];
  aclDataType weightDtype = dtypesInfo[1];
  aclDataType biasDtype = dtypesInfo[2];
  aclDataType scaleDtype = dtypesInfo[3];
  aclDataType outputDtype = dtypesInfo[4];
  // Create the input aclTensor.
  ret = CreateAclTensor(inputData, shapeInput, &deviceDataA, inputDtype, &input);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> inputTensorPtr(input, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataAPtr(deviceDataA, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the weight aclTensor.
  ret = CreateAclTensor(weightData, shapeWeight, &deviceDataB, weightDtype, &weight);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> weightTensorPtr(weight, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataBPtr(deviceDataB, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

    // Create scale.
  ret = CreateAclTensorND(scaleData, shapeScale, &deviceDataScale, scaleDtype, &scale);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> scaleTensorPtr(scale, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataScalePtr(deviceDataScale, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

    // Create bias.
  ret = CreateAclTensorND(biasData, shapeBias, &deviceDataBias, biasDtype, &bias);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> biasTensorPtr(bias, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataBiasPtr(deviceDataBias, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the output aclTensor.
  ret = CreateAclTensor(outputData, shapeResult, &deviceDataResult, outputDtype, &result);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> outputTensorPtr(result, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataResultPtr(deviceDataResult, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  aclIntArray *strides = aclCreateIntArray(convStrides.data(), 3);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> stridesPtr(strides, aclDestroyIntArray);
  CHECK_FREE_RET(strides != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *pads = aclCreateIntArray(convPads.data(), 3);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> padsPtr(pads, aclDestroyIntArray);
  CHECK_FREE_RET(pads != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *outPads = aclCreateIntArray(convOutPads.data(), 3);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> outPadsPtr(outPads, aclDestroyIntArray);
  CHECK_FREE_RET(outPads != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *dilations = aclCreateIntArray(convDilations.data(), 3);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> dilationsPtr(dilations, aclDestroyIntArray);
  CHECK_FREE_RET(dilations != nullptr, return ACL_ERROR_INTERNAL_ERROR);

  // 3. Call the CANN operator library API. Replace it with the specific API.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  void* workspaceAddr = nullptr;
  void* workspaceAddrNz = nullptr;

  int64_t* dstShape = nullptr;
  uint64_t dstShapeSize = 0;
  int actualFormat;
  // Convert weight to FRACTAL_Z_3D.
  // Compute the shape and format of the target tensor.
  ret = aclnnNpuFormatCastCalculateSizeAndFormat(weight, 33, -1, &dstShape, &dstShapeSize, &actualFormat);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnNpuFormatCastCalculateSizeAndFormat failed. ERROR: %d\n", ret); return ret);
  std::vector<int8_t> weightNzData(GetShapeSize(shapeWeight), 1);
  ret = CreateAclTensorWithFormat(weightNzData, shapeWeight, &dstShape, &dstShapeSize, &deviceDataBNz, weightDtype, &weightNz, static_cast<aclFormat>(actualFormat));
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("CreateAclTensorWithFormat failed. ERROR: %d\n", ret); return ret);

  // Call the first-phase API of aclnnNpuFormatCastGetWorkspaceSize.
  ret = aclnnNpuFormatCastGetWorkspaceSize(weight, weightNz, &workspaceSize, &executor);

  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnNpuFormatCastGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Allocate device memory based on workspaceSize computed by the first-phase API.
  if (workspaceSize > 0) {
      ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
      CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnNpuFormatCastGetWorkspaceSize.
  ret = aclnnNpuFormatCast(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnNpuFormatCast failed. ERROR: %d\n", ret); return ret);
  uint64_t workspaceSizeNz = 0;
  // Call the first-phase API of aclnnConvolutionWeightNz.
  ret = aclnnQuantConvolutionWeightNzGetWorkspaceSize(input, weightNz, bias, scale, nullptr, strides, pads, dilations,
                                              false, outPads, 1, 0, nullptr, result, &workspaceSizeNz, &executor);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT(
    "aclnnQuantConvolutionWeightNzGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Allocate device memory based on the workspaceSize computed by the first-phase API.

  std::unique_ptr<void, aclError (*)(void *)> workspaceAddrPtr(nullptr, aclrtFree);
  if (workspaceSizeNz > 0) {
    ret = aclrtMalloc(&workspaceAddrNz, workspaceSizeNz, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    workspaceAddrPtr.reset(workspaceAddrNz);
  }
  // Call the second-phase API of aclnnConvolutionWeightNz.
  ret = aclnnQuantConvolutionWeightNz(workspaceAddrNz, workspaceSizeNz, executor, stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnQuantConvolutionWeightNz failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronize and wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify this based on the specific API definition.
  auto size = GetShapeSize(shapeResult);
  std::vector<uint16_t> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), deviceDataResult,
                    size * sizeof(uint16_t), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  int64_t printSize = size > 10 ? 10 : size;
  for (int64_t i = 0; i < printSize; i++) {
    LOG_PRINT("result[%ld] is: %d\n", i, resultData[i]);
  }

  return ACL_SUCCESS;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. For details, see the ACL API manual.
  // Fill in the deviceId based on your actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  std::vector<aclDataType> dtypesInfo = {aclDataType::ACL_INT8, aclDataType::ACL_INT8, aclDataType::ACL_FLOAT,
    aclDataType::ACL_FLOAT, aclDataType::ACL_BF16}; // The data types of input, weight, bias, scale, and output respectively.
  auto ret = aclnnQuantConvolutionWeightNzTest(deviceId, stream, dtypesInfo);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnQuantConvolutionWeightNzTest failed. ERROR: %d\n", ret); return ret);

  Finalize(deviceId, stream);
  return 0;
}