aclnnConvTbc

📄 View Source Code

Applicable Products

Function

• Description: Implements one-dimensional convolution with input and output dimensions of T (time or spatial dimension), B (batch), and C (channel).

• Formula:

  Assume the shape of the input (self) is ($H_{\text{in}},N,C_{\text{in}}$) and the shape of the output (out) is ($H_{\text{out}},N,C_{\text{out}}$), then the output is expressed as:

  $$out_{N_i,C_{out j}} = bias(C_{out j}) + \sum_{k = 0}^{C_{in} - 1} weight(k, C_{out j}) \cdot self(N_i, k)$$

  Where $N$ represents the batch size, $C$ represents the number of channels, and $H$ represents the time or spatial dimension.

Function Prototype

Each operator is divided into a two-phase API. You must first call the aclnnConvTbcGetWorkspaceSize API to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call the aclnnConvTbc API to perform the computation.

aclnnStatus aclnnConvTbcGetWorkspaceSize(
    const aclTensor       *self,
    const aclTensor       *weight,
    const aclTensor       *bias,
    const int64_t          pad,
    aclTensor             *out,
    int8_t                 cubeMathType,
    uint64_t              *workspaceSize,
    aclOpExecutor         **executor)

aclnnStatus aclnnConvTbc(
    void            *workspace,
    const uint64_t   workspaceSize,
    aclOpExecutor   *executor,
    aclrtStream     stream)

aclnnConvTbcGetWorkspaceSize

• Parameters

  Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-contiguous Tensor
  self (aclTensor*)	Input	self in the formula, represents the convolution input.	Supports empty tensors. The data type must comply with the data type deduction rules with the data type of weight (see deduction relationship). The shape is (N, Cin, Hin). N ≥ 0, C ≥ 1, H ≥ 0.	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8	ND, NCL	3	√
  weight (aclTensor*)	Input	Represents the convolution weight in the formula.	Supports empty tensors. The data type must satisfy the data type deduction rules with the data type of self (see Deduction Relationship). The shape is (Cout, Cin, K). All dimensions are ≥ 1.	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8	ND, NCL	3	√
  bias (aclTensor*)	Input	bias in the formula, representing the convolution bias.	The data type is the same as that of self and weight. It is one-dimensional and equal to the first dimension of weight. Null pointers are not allowed.	FLOAT, FLOAT16, BFLOAT16	ND	1	√
  pad (int64_t)	Input	Specifies the number of padding elements on the left and right sides of the H dimension.	The value must be in the range [0, 255].	INT64	-	-	-
  out (aclTensor*)	Output	out in the formula, representing the convolution output.	Supports empty tensors. The data type is the same as that of self. The shape is (N, Cout, Hout). The number of channels equals the first dimension of weight, and other dimensions are ≥ 0.	FLOAT, FLOAT16, BFLOAT16, HIFLOAT8	ND, NCL	3	√
  cubeMathType (int8_t)	Input	Determines which computation logic the Cube unit should use for calculation.	0 (KEEP_DTYPE): Retains the input data type for computation. 1 (ALLOW_FP32_DOWN_PRECISION): Allows FLOAT reduced-precision computation to improve performance. 2 (USE_FP16): Uses FLOAT16 precision for computation. 3 (USE_HF32): Uses HFLOAT32 (mixed precision) for computation.	INT8	-	-	-
  workspaceSize (uint64_t*)	Output	Returns the workspace size to be requested on the Device side.	-	-	-	-	-
  executor (aclOpExecutor**)	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

• Return Value

  aclnnStatus: Return status code. For details, see aclnn Return Code.

  The first-phase API performs input parameter validation and returns an error in the following scenarios:

  Return Value	Error Code	Description
  ACLNN_ERR_PARAM_NULLPTR	161001	The pointer-type input parameter passed in is a null pointer.
  ACLNN_ERR_PARAM_INVALID	161002	The data type and format of self, weight, bias, or out are not within the supported range.
  		The data types of self, weight, and out are inconsistent.
  		The shape of out does not meet the infershape result.
  		The shape of out contains values less than 0.
  		The dim of self is not 3.
  		The dim of weight is not 3.
  		The dim of bias is not 1.
  		The value of the second dimension of self is not equal to the value of the second dimension of weight.
  		The value of bias is not equal to the first dimension of weight.
  ACLNN_ERR_INNER_NULLPTR	561103	Internal API validation error, usually caused by input data or attribute specifications being outside the supported range.
  ACLNN_ERR_RUNTIME_ERROR	361001	An exception occurs when the API calls an NPU runtime interface, for example, SocVersion is not supported.

aclnnConvTbc

• Parameters

  Parameter	Input/Output	Description
  workspace	Input	Workspace memory address applied for on the Device side.
  workspaceSize	Input	Workspace size applied for on the Device side, obtained through the first-phase API aclnnConvTbcGetWorkspaceSize.
  executor	Input	Operator executor that contains the operator computation flow.
  stream	Input	Specifies the stream for task execution.

• Return Value

  aclnnStatus: Return status code. For details, see aclnn Return Code.

Constraints

• Deterministic computation

  • aclnnConvTbc defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <memory>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_convolution.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define CHECK_FREE_RET(cond, return_expr) \
  do {                                    \
    if (!(cond)) {                        \
      Finalize(deviceId, stream);         \
      return_expr;                        \
    }                                     \
  } while (0)

#define LOG_PRINT(message, ...)      \
  do {                               \
    printf(message, ##__VA_ARGS__);  \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shape_size = 1;
  for (auto i: shape) {
    shape_size *= i;
  }
  return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor, aclFormat dataFormat) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for memory on the device side.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

  // Call aclrtMemcpy to copy data from the host side to the device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, dataFormat,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

void Finalize(int32_t deviceId, aclrtStream& stream)
{
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
}

int aclnnConvTbcTest(int32_t deviceId, aclrtStream& stream)
{
  auto ret = Init(deviceId, &stream);
  // Handle the check based on your own needs.
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct inputs and outputs. Customize based on the API interface.
  std::vector<int64_t> shapeInput = {2, 2, 3};
  std::vector<int64_t> shapeWeight = {3, 3, 4};
  std::vector<int64_t> shapeBias = {4};
  std::vector<int64_t> shapeResult = {2, 2, 4};

  void* deviceDataA = nullptr;
  void* deviceDataB = nullptr;
  void* deviceDataC = nullptr;
  void* deviceDataResult = nullptr;

  aclTensor* input = nullptr;
  aclTensor* weight = nullptr;
  aclTensor* bias = nullptr;
  aclTensor* result = nullptr;
  std::vector<float> inputData(GetShapeSize(shapeInput), 1);
  std::vector<float> weightData(GetShapeSize(shapeWeight), 1);
  std::vector<float> biasData(GetShapeSize(shapeBias), 1);
  std::vector<float> outputData(GetShapeSize(shapeResult), 1);

  // Create the input aclTensor.
  ret = CreateAclTensor(inputData, shapeInput, &deviceDataA, aclDataType::ACL_FLOAT, &input, aclFormat::ACL_FORMAT_NCL);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> inputTensorPtr(input, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataAPtr(deviceDataA, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the weight aclTensor.
  ret = CreateAclTensor(weightData, shapeWeight, &deviceDataB, aclDataType::ACL_FLOAT, &weight, aclFormat::ACL_FORMAT_NCL);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> weightTensorPtr(weight, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataBPtr(deviceDataB, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the bias aclTensor.
  ret = CreateAclTensor(biasData, shapeBias, &deviceDataC, aclDataType::ACL_FLOAT, &bias, aclFormat::ACL_FORMAT_ND);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> biasTensorPtr(bias, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataCPtr(deviceDataC, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // Create the result aclTensor.
  ret = CreateAclTensor(outputData, shapeResult, &deviceDataResult, aclDataType::ACL_FLOAT, &result, aclFormat::ACL_FORMAT_NCL);
  std::unique_ptr<aclTensor, aclnnStatus (*)(const aclTensor *)> outputTensorPtr(result, aclDestroyTensor);
  std::unique_ptr<void, aclError (*)(void *)> deviceDataResultPtr(deviceDataResult, aclrtFree);
  CHECK_FREE_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Replace with the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the aclnnConvTbc first-phase API.
  ret = aclnnConvTbcGetWorkspaceSize(input, weight, bias, 1, result, 1, &workspaceSize, &executor);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvTbcGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  std::unique_ptr<void, aclError (*)(void *)> workspaceAddrPtr(nullptr, aclrtFree);
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
    workspaceAddrPtr.reset(workspaceAddr);
  }
  // Call the aclnnConvTbc second-phase API.
  ret = aclnnConvTbc(workspaceAddr, workspaceSize, executor, stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvTbc failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronize and wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.
  auto size = GetShapeSize(shapeResult);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), deviceDataResult,
                    size * sizeof(float), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  return ACL_SUCCESS;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
  // Enter the deviceId based on your actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = aclnnConvTbcTest(deviceId, stream);
  CHECK_FREE_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnConvTbcTest failed. ERROR: %d\n", ret); return ret);

  Finalize(deviceId, stream);
  return 0;
}