aclnnConvolution

📄 View Source Code

Applicable Products

Function

• Description: Implements the convolution function, supporting 1D convolution, 2D convolution, and 3D convolution, as well as transposed convolution, dilated convolution, and grouped convolution.

  When the input parameter transposed = True, it indicates the use of transposed convolution or fractionally-strided convolution. It can be viewed as the gradient or inverse operation of a regular convolution, i.e., restoring the output shape of a convolution back to the input shape while maintaining a connectivity pattern compatible with the convolution. Its parameters are similar to those of a regular convolution, including input channels, output channels, kernel size, stride, padding, output padding, groups, bias, dilation, and so on.

• Formula:

  Assume the input shape is $(N, C_{\text{in}}, D, H, W)$, the weight shape is $(C_{\text{out}}, C_{\text{in}}, K_d, K_h, K_w)$, and the output shape is $(N, C_{\text{out}}, D_{\text{out}}, H_{\text{out}}, W_{\text{out}})$, where $N$ denotes the batch size, $C$ is the number of channels, $D$, $H$, and $W$ are the depth, height, and width of the sample respectively, and $K_d$, $K_h$, and $K_w$ are the depth, height, and width of the convolution kernel respectively. The output is expressed as:

  $$\text{output}(N_i, C_{\text{out}_j}, D_{\text{out}}, H_{\text{out}}, W_{\text{out}}) = \sum_{k = 0}^{C_{\text{in}} - 1} \text{weight}(C_{\text{out}_j}, k) \star \text{input}(N_i, k) + \text{bias}(C_{\text{out}_j})$$

  Where $\star$ denotes the convolution operation, which depends on the convolution input dimensions and the convolution type (dilated convolution, grouped convolution). $N$ represents the batch size, $C$ represents the number of channels, and $D$, $H$, and $W$ represent the depth, height, and width respectively. The formulas for the corresponding output dimensions are as follows:

  • When the input parameter transposed = False:

    $$D_{\text{out}}=[(D + 2 \times padding[0] - dilation[0] \times (K_d - 1) - 1 ) / stride[0]] + 1 \\ H_{\text{out}}=[(H + 2 \times padding[1] - dilation[1] \times (K_h - 1) - 1 ) / stride[1]] + 1 \\ W_{\text{out}}=[(W + 2 \times padding[2] - dilation[2] \times (K_w - 1) - 1 ) / stride[2]] + 1$$

  • When the input parameter transposed = True:

    $$D_{\text{out}}=(D - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{dilation}[0] \times (K_d - 1) + \text {outputPadding}[0] + 1 \\ H_{\text{out}}=(H - 1) \times \text{stride}[1] - 2 \times \text{padding}[1] + \text{dilation}[1] \times (K_h - 1) + \text {outputPadding}[1] + 1 \\ W_{\text{out}}=(W - 1) \times \text{stride}[2] - 2 \times \text{padding}[2] + \text{dilation}[2] \times (K_w - 1) + \text {outputPadding}[2] + 1$$

Function Prototype

Each operator is divided into a two-phase API. You must first call the aclnnConvolutionGetWorkspaceSize API to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call the aclnnConvolution API to perform the computation.

aclnnStatus aclnnConvolutionGetWorkspaceSize(
    const aclTensor       *input,
    const aclTensor       *weight,
    const aclTensor       *bias,
    const aclIntArray     *stride,
    const aclIntArray     *padding,
    const aclIntArray     *dilation,
    bool                   transposed,
    const aclIntArray     *outputPadding,
    const int64_t          groups,
    aclTensor             *output,
    int8_t                 cubeMathType,
    uint64_t              *workspaceSize,
    aclOpExecutor         **executor)

aclnnStatus aclnnConvolution(
    void            *workspace,
    const uint64_t   workspaceSize,
    aclOpExecutor   *executor,
    aclrtStream      stream)

aclnnConvolutionGetWorkspaceSize

• Parameters

  Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-Contiguous Tensor
  input (aclTensor*)	Input	In the formula, input represents the convolution input.	Supports empty tensors. The data type must satisfy the data type deduction rules with weight (see Deduction Relationship and Constraints).	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8, FLOAT8_E4M3FN	NCL, NCHW, NCDHW	3-5	√
  weight (aclTensor*)	Input	Weight in the formula, representing the convolution weight.	Supports empty tensors. The data type must satisfy the data type deduction rules with input (see Deduction Relationship and Constraints).	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8, FLOAT8_E4M3FN	NCL, NCHW, NCDHW	3-5	√
  bias (aclTensor*)	Input	Bias in the formula, representing the convolution bias.	For scenarios without bias, pass nullptr. When transposed=false, it is one-dimensional and its value equals the first dimension of weight. When transposed=true, it is one-dimensional and its value equals weight.shape[1] * groups. Only the ND format is supported.	FLOAT, FLOAT16, BFLOAT16	ND, NCL, NCHW, NCDHW	1-5	√
  stride (aclIntArray*)	Input	Convolution stride.	The array length must equal the number of input dimensions minus 2, and each value must be greater than 0.	INT64	-	-	-
  padding (aclIntArray*)	Input	Padding for the input.	Array Length: 1 or 2 for non-transposed conv1d; 1 for transposed conv1d; 2 or 4 for conv2d; 3 for conv3d. The value must be greater than or equal to 0.	INT64	-	-	-
  dilation (aclIntArray*)	Input	Spacing between elements in the convolution kernel.	The array length must equal the number of input dimensions minus 2, and each value must be greater than 0.	INT64	-	-	-
  transposed (bool)	Input	Indicates whether it is a transposed convolution.	-	BOOL	-	-	-
  outputPadding (aclIntArray*)	Input	Specifies the padding for all edges of the output in the case of transposed convolution.	This configuration is ignored in the case of non-transposed convolution. The array length must equal the number of input dimensions minus 2. The value must be greater than or equal to 0 and less than the value of the corresponding dimension of stride or dilation.	INT32	-	-	-
  groups (int64_t)	Input	Indicates the number of block connections from the input channel to the output channel.	The value must be in the range [1, 65535] and satisfy groups * weight C dimension = input C dimension.	INT64	-	-	-
  output (aclTensor*)	Output	Represents the convolution output, denoted as out in the formula.	Supports empty tensors. The data type must be consistent with the data type derived from input and weight. The number of channels equals the first dimension of weight, and other dimensions are ≥ 0.	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8, FLOAT8_E4M3FN	NCL, NCHW, NCDHW	3-5	√
  cubeMathType (int8_t)	Input	Determines which computation logic the Cube unit uses for the operation.	If the input data types have a deduction relationship, this parameter defaults to processing the data type after deduction. The supported enumeration values are as follows: 0 (KEEP_DTYPE): Keeps the input data type for computation. 1 (ALLOW_FP32_DOWN_PRECISION): Allows FLOAT to reduce precision for computation, improving performance. 2 (USE_FP16): Uses FLOAT16 precision for computation. 3 (USE_HF32): Uses HFLOAT32 (mixed precision) for computation.	INT8	-	-	-
  workspaceSize（uint64_t*）	Output	Returns the workspace size to be requested on the Device side.	-	-	-	-	-
  executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

• Return Value

  aclnnStatus: Return status code. For details, see aclnn Return Code.

  The first-phase API performs input parameter validation and returns an error in the following scenarios:

  Return Value	Return Code	Description
  ACLNN_ERR_PARAM_NULLPTR	161001	The pointer-type input parameter is a null pointer.
  ACLNN_ERR_PARAM_INVALID	161002	The data type or data format of input, weight, bias, or output is not within the supported range.
  		The data types of input and output are inconsistent. When transposed=false, inconsistent data types between input and output are supported and do not trigger this error.
  		The input shapes of stride, padding, dilation, or outputPadding are incorrect.
  		Incorrect groups input.
  		The output shape does not match the infershape result.
  		The outputPadding value does not meet the requirements.
  		The empty tensors passed as input, weight, bias, or output, where some dimensions are zero, do not meet the requirements.
  		The input spatial size after padding is smaller than the weight spatial size (after dilation expansion, if dilation > 1) in non-transpose mode.
  		In transpose mode, the shape of bias is not 1.
  		The stride or dilation is less than 0, which does not meet the requirement.
  		The current processor does not support convolution.
  ACLNN_ERR_INNER_NULLPTR	561103	An internal API validation error occurs, typically because the input data or attribute specifications are outside the supported range.
  ACLNN_ERR_RUNTIME_ERROR	361001	An exception occurs when the API calls an NPU runtime interface, for example, SocVersion is not supported.

aclnnConvolution

• Parameters

  Parameter	Input/Output	Description
  workspace	Input	Workspace memory address applied for on the Device side.
  workspaceSize	Input	Workspace size applied for on the Device side, obtained from the first-phase API aclnnConvolutionGetWorkspaceSize.
  Executor	Input	Operator executor, which contains the operator computation process.
  Stream	Input	Specifies the stream that executes the task.

• Return Value

  aclnnStatus: Return status code. For details, see aclnn Return Code.

Constraints

• deterministic computation

  • aclnnConvolution defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <memory>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_convolution.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define CHECK_FREE_RET(cond, return_expr) \
  do {                                    \
    if (!(cond)) {                        \
      Finalize(deviceId, stream);         \
      return_expr;                        \
    }                                     \
  } while (0)

#define LOG_PRINT(message, ...)      \
  do {                               \
    printf(message, ##__VA_ARGS__);  \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shape_size = 1;
  for (auto i: shape) {
    shape_size *= i;
  }
  return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate for resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for device memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Copy data from the host side to the device memory using aclrtMemcpy.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Compute the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Create an aclTensor by calling the aclCreateTensor API.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_NCHW,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

void Finalize(int32_t deviceId, aclrtStream& stream)
{
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
}

int aclnnConvolutionTest(int32_t deviceId, aclrtStream& stream)
{
  auto ret = Init(deviceId, &stream);
  // Handle the check result as needed.
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API interface.
  std::vector<int64_t> shapeInput = {2, 2, 2, 2};
  std::vector<int64_t> shapeWeight = {1, 2, 1, 1};
  std::vector<int64_t> shapeResult = {2, 1, 2, 2};
  std::vector<int64_t> convStrides;
  std::vector<int64_t> convPads;
  std::vector<int64_t> convOutPads;
  std::vector<int64_t> convDilations;

  void* deviceDataA = nullptr;
  void* deviceDataB = nullptr;
  void* deviceDataResult = nullptr;

  aclTensor* input = nullptr;
  aclTensor* weight = nullptr;
  aclTensor* result = nullptr;
  std::vector<float> inputData(GetShapeSize(shapeInput), 1);
  std::vector<float> weightData(GetShapeSize(shapeWeight), 1);
  std::vector<float> outputData(GetShapeSize(shapeResult), 1);
  convStrides = {1, 1, 1, 1};
  convPads = {0, 0, 0, 0};
  convOutPads = {0, 0, 0, 0};
  convDilations = {1, 1, 1, 1};

  // Create the input aclTensor.
  ret = CreateAclTensor(inputData, shapeInput, &deviceDataA, aclDataType::ACL_FLOAT, &input);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> inputTensorPtr(input, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataAPtr(deviceDataA, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the weight aclTensor.
  ret = CreateAclTensor(weightData, shapeWeight, &deviceDataB, aclDataType::ACL_FLOAT, &weight);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> weightTensorPtr(weight, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataBPtr(deviceDataB, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the output aclTensor.
  ret = CreateAclTensor(outputData, shapeResult, &deviceDataResult, aclDataType::ACL_FLOAT, &result);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> outputTensorPtr(result, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataResultPtr(deviceDataResult, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  aclIntArray *strides = aclCreateIntArray(convStrides.data(), 2);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> stridesPtr(strides, aclDestroyIntArray);
  CHECK_FREE_RET(strides != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *pads = aclCreateIntArray(convPads.data(), 2);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> padsPtr(pads, aclDestroyIntArray);
  CHECK_FREE_RET(pads != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *outPads = aclCreateIntArray(convOutPads.data(), 2);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> outPadsPtr(outPads, aclDestroyIntArray);
  CHECK_FREE_RET(outPads != nullptr, return ACL_ERROR_INTERNAL_ERROR);
  aclIntArray *dilations = aclCreateIntArray(convDilations.data(), 2);
  std::unique_ptr<aclIntArray, aclnnStatus (*)(const aclIntArray *)> dilationsPtr(dilations, aclDestroyIntArray);
  CHECK_FREE_RET(dilations != nullptr, return ACL_ERROR_INTERNAL_ERROR);

  // 3. Call the CANN operator library API. Replace with the specific API.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnConvolution.
  ret = aclnnConvolutionGetWorkspaceSize(input, weight, nullptr, strides, pads, dilations, false, outPads, 1, result, 1,
                                         &workspaceSize, &executor);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvolutionGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  std::unique_ptr<void, aclError (*)(void *)> workspaceAddrPtr(nullptr, aclrtFree);
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    workspaceAddrPtr.reset(workspaceAddr);
  }
  // Call the second-phase API of aclnnConvolution.
  ret = aclnnConvolution(workspaceAddr, workspaceSize, executor, stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvolution failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronously wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify this based on the specific API definition.
  auto size = GetShapeSize(shapeResult);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), deviceDataResult,
                    size * sizeof(float), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  return ACL_SUCCESS;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. Refer to the ACL API manual.
  // Fill in the deviceId based on the actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = aclnnConvolutionTest(deviceId, stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvolutionTest failed. ERROR: %d\n", ret); return ret);

  Finalize(deviceId, stream);
  return 0;
}