Documentation
Download
文档中心
aclnnAddbmm&aclnnInplaceAddbmm

aclnnAddbmm&aclnnInplaceAddbmm

Applicable Products

ProductSupported
Ascend 950PR/Ascend 950DT√
Atlas A3 training products/Atlas A3 inference products√
Atlas A2 training products/Atlas A2 inference products√
Atlas 200I/500 A2 inference products×
Atlas inference products√
Atlas training products√

Function

  • Description: First performs matrix multiplication of batch1 and batch2, then adds the results in batches along the first dimension (batch dimension), compressing the 3D vector into a 2D vector (with the shape size being the last two dimensions). The result is then multiplied by α, and summed with the product of β and self to obtain the final result.

  • Formula:

    out=βself+α(∑i=0b−1batch1i@batch2i)out = βself+α(\sum_{i=0}^{b-1} batch1_{i}@batch2_{i})

    Note: If β is 0, self is ignored and does not participate in the calculation. If α is 0, batch1 and batch2 are ignored and do not participate in the calculation.

  • Example:

The shape of self is [3,5], the shape of batch1 is [10,3,4], the shape of batch2 is [10,4,5], and the shape of the computed output out is [3,5].

Function Prototype

  • aclnnAddbmm and aclnnInplaceAddbmm implement the same functionality. The differences are as follows. Select the appropriate operator based on your actual scenario.

    • aclnnAddbmm: Requires creating a new output tensor object to store the computation result.

    • aclnnInplaceAddbmm: Does not require creating a new output tensor object. The computation result is stored directly in the memory of the input tensor.

  • Each operator is divided into a two-phase API. You must first call the "aclnnAddbmmGetWorkspaceSize" or "aclnnInplaceAddbmmGetWorkspaceSize" API to obtain the required workspace size and the executor that contains the operator computation process, and then call the "aclnnAddbmm" or "aclnnInplaceAddbmm" API to execute the computation.

aclnnStatus aclnnAddbmmGetWorkspaceSize(
  const aclTensor     *self,
  const aclTensor     *batch1,
  const aclTensor     *batch2,
  const aclScalar     *beta,
  const aclScalar     *alpha,
  aclTensor           *out,
  int8_t               cubeMathType,
  uint64_t            *workspaceSize,
  aclOpExecutor       **executor)
aclnnStatus aclnnAddbmm(
  void              *workspace,
  uint64_t           workspaceSize,
  aclOpExecutor     *executor,
  aclrtStream        stream)
aclnnStatus aclnnInplaceAddbmmGetWorkspaceSize(
  aclTensor         *selfRef,
  const aclTensor   *batch1,
  const aclTensor   *batch2,
  const aclScalar   *beta,
  const aclScalar   *alpha,
  int8_t             cubeMathType,
  uint64_t          *workspaceSize,
  aclOpExecutor     **executor)
aclnnStatus aclnnInplaceAddbmm(
  void             *workspace,
  uint64_t          workspaceSize,
  aclOpExecutor    *executor,
  aclrtStream       stream)

aclnnAddbmmGetWorkspaceSize

  • Parameters:

    Parameter Input/Output Description Instruction Data Type Data Format Dimension (Shape) Non-contiguous Tensor
    self Input Self in the formula.
    • The data type must match that of out.
    • The shape must satisfy the broadcast relationship with the last two dimensions of batch1@batch2 (note: only self can be broadcast to the same shape as the last two dimensions of batch1@batch2; the last two dimensions of batch1@batch2 cannot be broadcast to match self. For example, self:[3,5], batch1@batch2:[1,1,1] will fail).
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 1-2 dimensions √
    batch1 Input Input batch1 in the formula.
    • The data type must satisfy the data type deduction rules with the data types of self and batch2 (see Deduction Relationship and Constraints).
    • The shape only supports three dimensions and the first dimension batch1 must satisfy the broadcast relationship with the batch dimension of batch2.
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 3 √
    batch2 Input Input batch2 in the formula.
    • The data type must satisfy the data type deduction rules with the data types of self and batch1 (see Deduction Relationship and Constraints).
    • The shape only supports three dimensions and the first dimension batch must satisfy the broadcast relationship with the batch dimension of batch1.
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 3 √
    beta Input. Input β in the formula. The data type needs to be convertible to the data type derived from self and batch1@batch2 (see Mutual Derivation Relationship and Constraints). FLOAT, FLOAT16, DOUBLE, INT32, INT64, INT16, INT8, UINT8, BOOL - - -
    alpha Input Input α in the formula. The data type needs to be convertible to the data type derived from self and batch1@batch2 (see Deduction Relationship and Constraints). FLOAT, FLOAT16, DOUBLE, INT32, INT64, INT16, INT8, UINT8, BOOL - - -
    out Output Output out in the formula.
    • The data type must match self.
    • The shape must match the last two dimensions of batch1@batch2.
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 2 √
    cubeMathType Input Specifies the computation logic of the Cube unit. If the input data types have a deduction relationship, this parameter defaults to processing the data type after deduction. The supported enumeration values are as follows:
    • 0: KEEP_DTYPE, keeps the input data type for calculation.
    • 1: ALLOW_FP32_DOWN_PRECISION, supports input precision reduction for calculation.
    • 2: USE_FP16, supports reducing input precision to FLOAT16 for calculation.
    • 3: USE_HF32, supports reducing input precision to HFLOAT32 for calculation.
    		 INT8 - - -
    workspaceSize Output Parameter Returns the workspace size to be applied on the Device side. - - - - -
    executor Output Parameter Returns the operator executor, which includes the operator calculation process. - - - - -

    • Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products, Ascend 950PR/Ascend 950DT:

      • cubeMathType=1: When the input data type is FLOAT32, it will be converted to HFLOAT32 for calculation. When the input is of other data types, it is not processed.

      • cubeMathType=2: When the input data type is FLOAT32, it will be converted to FLOAT16 for calculation. When the input is of other data types, it is not processed.

      • cubeMathType=3: When the input data type is FLOAT32, it will be converted to HFLOAT32 for calculation. When the input is of other data types, it is not processed.

    • Atlas training products, Atlas inference products:

      • BFLOAT16 data type is not supported;

      • When the input data type is FLOAT32, cubeMathType=0 is not supported;

      • cubeMathType=1: when the input data type is FLOAT32, it will be converted to FLOAT16 for calculation; when the input is of other data types, no processing is performed;

      • cubeMathType=3 is not supported.

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

    The first-phase API performs input parameter validation and returns an error in the following scenarios:

    Return Value Error Code Description
    ACLNN_ERR_PARAM_NULLPTR 161001 Indicates that self, batch1, or batch2 is a null pointer.
    ACLNN_ERR_PARAM_INVALID 161002 The data type of self, batch1, batch2, or out is not within the supported range.
    The data type of self, batch1, batch2, or out does not satisfy the derivation rules.
    self cannot be broadcast with batch1@batch2.
    The first dimensions of batch1 and batch2 are not equal or cannot be broadcast.
    The dimensions of batch1 and batch2 are not three-dimensional.
    The last dimension of batch1 is not equal to the penultimate dimension of batch2.
    self and batch1@batch2 must both be empty tensors or both be non-empty tensors.

aclnnAddbmm

  • Parameters:

    Parameter Input/Output Description
    workspace Input The workspace memory address applied on the Device side.
    workspaceSize Input The workspace size applied on the Device side, obtained from the first-phase API aclnnAddbmmGetWorkspaceSize.
    executor Input Operator executor, which includes the calculation process of the operator.
    stream Input Specifies the stream for executing the task.

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

aclnnInplaceAddbmmGetWorkspaceSize

  • Parameters:

    Parameter Input/Output Description Instruction Data Type Data Format Dimension (Shape) Non-contiguous Tensor
    selfRef Input|Output Input/output tensor, i.e., the input self and out in the formula.
    • The data type and the data type of batch1@batch2 must satisfy the data type derivation rules (see Deduction Relationship and Constraints).
    • The shape must be consistent with the last two dimensions of batch1@batch2.
    • Supports empty tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 2 √
    batch1 Input Input batch1 in the formula.
    • The data type and the data types of self and batch2 must satisfy the data type derivation rules (see Deduction Relationship and Constraints).
    • The shape only supports three dimensions and the first dimension batch1 must satisfy the broadcast relationship with the batch dimension of batch2.
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 3 √
    batch2 Input Input batch2 in the formula.
    • The data type must satisfy the data type derivation rules with the data types of self and batch1 (see Deduction Relationship and Constraints).
    • The shape only supports three dimensions and the first dimension batch must satisfy the broadcast relationship with the batch dimension of batch1.
    • Supports empty Tensor.
    		 BFLOAT16, FLOAT16, FLOAT32 ND 3 √
    beta Input Input β in the formula. The data type needs to be convertible to the data type derived from self and batch1@batch2 (see Deduction Relationship and Constraints). FLOAT, FLOAT16, DOUBLE, INT32, INT64, INT16, INT8, UINT8, BOOL - - -
    alpha Input In the formula, the input α. The data type needs to be convertible to the data type derived from self and batch1@batch2 (see Deduction Relationship and Constraints). FLOAT, FLOAT16, DOUBLE, INT32, INT64, INT16, INT8, UINT8, BOOL - - -
    cubeMathType Input Specifies the computation logic of the Cube unit. If the input data types have a deduction relationship, this parameter defaults to processing the data type after deduction. The supported enumeration values are as follows:
    • 0: KEEP_DTYPE, keeps the input data type for computation.
    • 1: ALLOW_FP32_DOWN_PRECISION, supports precision reduction of the input data for computation.
    • 2: USE_FP16, supports reducing the input precision to FLOAT16 for computation.
    • 3: USE_HF32, supports reducing the input precision to HFLOAT32 for computation.
    		 INT8 - - -
    workspaceSize Output Parameter Returns the workspace size to be applied on the device side. - - - - -
    Executor Output Parameter Returns the op executor, which includes the calculation process of the operator. - - - - -

    • Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products, Ascend 950PR/Ascend 950DT:

      • cubeMathType=1: When the input data type is FLOAT32, it will be converted to HFLOAT32 for calculation. When the input is of other data types, no processing is performed.

      • cubeMathType=2: When the input data type is FLOAT32, it will be converted to FLOAT16 for calculation. When the input is of other data types, no processing is performed.

      • cubeMathType=3: When the input data type is FLOAT32, it will be converted to HFLOAT32 for calculation; when the input is of other data types, no processing is performed.

    • Atlas training products, Atlas inference products:

      • BFLOAT16 data type is not supported.

      • cubeMathType=0 is not supported when the input data type is FLOAT32.

      • cubeMathType=1: When the input data type is FLOAT32, it will be converted to FLOAT16 for calculation; when the input is of other data types, no processing is performed.

      • cubeMathType=3 is not supported.

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

    The first-phase API performs input parameter validation and returns an error in the following scenarios:

    Return Value Error Code Description
    ACLNN_ERR_PARAM_NULLPTR 161001 selfRef, batch1, or batch2 passed in is a null pointer.
    ACLNN_ERR_PARAM_INVALID 161002 The data type of selfRef, batch1, or batch2 is not within the supported range.
    The data type of selfRef, batch1, or batch2 does not satisfy the Derivation Rules.
    selfRef cannot be broadcast with batch1@batch2.
    The first dimensions of batch1 and batch2 are not equal or cannot be broadcast.
    The dimensions of batch1 and batch2 are not three-dimensional.
    The last dimension of batch1 and the second-to-last dimension of batch2 are not equal.
    selfRef and batch1@batch2 must both be empty tensors or both be non-empty tensors.

aclnnInplaceAddbmm

  • Parameters:

    Parameter Input/Output Description
    workspace Input Memory address of the workspace applied on the Device side.
    workspaceSize Input Size of the workspace applied on the Device side, obtained from the first-phase API aclnnInplaceAddbmmGetWorkspaceSize.
    Executor Input Operator executor, which includes the operator calculation process.
    stream Input Specifies the stream for task execution.

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

  • Determinism Description:

    • Atlas training products, Atlas inference products: aclnnAddbmm&aclnnInplaceAddbmm default to a deterministic implementation.

    • Ascend 950PR/Ascend 950DT: aclnnAddbmm&aclnnInplaceAddbmm default to a deterministic implementation.

  • Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products: Data type deduction where one of the two inputs batch1 and batch2 is BFLOAT16 and the other is FLOAT or FLOAT16 is not supported.

  • Atlas training products and Atlas inference products: The Cube unit does not support FLOAT32 computation. When the input is FLOAT32, you can set cubeMathType=1 (ALLOW_FP32_DOWN_PRECISION) to allow the API to internally cast to FLOAT16 for computation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_addbmm.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: resource initialization.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to allocate memory on the device side.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Call aclrtMemcpy to copy data from the host side to the device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the acl API manual.
  // Set deviceId based on the actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct inputs and outputs. Custom construction is required based on the API.
  std::vector<int64_t> selfShape = {2, 4};
  std::vector<int64_t> batch1Shape = {1, 2, 3};
  std::vector<int64_t> batch2Shape = {1, 3, 4};
  std::vector<int64_t> outShape = {2, 4};
  void* selfDeviceAddr = nullptr;
  void* batch1DeviceAddr = nullptr;
  void* batch2DeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  aclTensor* self = nullptr;
  aclTensor* batch1 = nullptr;
  aclTensor* batch2 = nullptr;
  aclScalar* alpha = nullptr;
  aclScalar* beta = nullptr;
  aclTensor* out = nullptr;
  std::vector<float> selfHostData = {0, 1, 2, 3, 4, 5, 6, 7};
  std::vector<float> batch1HostData = {1, 1, 1, 2, 2, 2};
  std::vector<float> batch2HostData = {1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4};
  std::vector<float> outHostData(8, 0);
  int8_t cubeMathType = 1;
  float alphaValue = 1.2f;
  float betaValue = 1.0f;
  // Create the self aclTensor.
  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the batch1 aclTensor.
  ret = CreateAclTensor(batch1HostData, batch1Shape, &batch1DeviceAddr, aclDataType::ACL_FLOAT, &batch1);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the batch2 aclTensor.
  ret = CreateAclTensor(batch2HostData, batch2Shape, &batch2DeviceAddr, aclDataType::ACL_FLOAT, &batch2);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the alpha aclScalar.
  alpha = aclCreateScalar(&alphaValue,aclDataType::ACL_FLOAT);
  CHECK_RET(alpha != nullptr, return ret);
  // Create the beta aclScalar.
  beta = aclCreateScalar(&betaValue,aclDataType::ACL_FLOAT);
  CHECK_RET(beta != nullptr, return ret);
  // Create the out aclTensor.
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Replace with the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor = nullptr;
  // Call the first-phase API of aclnnAddbmm.
  ret = aclnnAddbmmGetWorkspaceSize(self, batch1, batch2, beta, alpha, out, cubeMathType, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddbmmGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnAddbmm.
  ret = aclnnAddbmm(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddbmm failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronize and wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host. Modify based on the specific API definition.
  auto size = GetShapeSize(outShape);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr,
                    size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  // aclnnInplaceAddbmm
  // Step 3: Call the CANN operator library API.
  LOG_PRINT("\ntest aclnnInplaceAddbmm\n");
  // Call the first-phase API of aclnnInplaceAddbmm.
  ret = aclnnInplaceAddbmmGetWorkspaceSize(self, batch1, batch2, beta, alpha, cubeMathType, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnInplaceAddbmmGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnInplaceAddbmm.
  ret = aclnnInplaceAddbmm(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnInplaceAddbmm failed. ERROR: %d\n", ret); return ret);

  // step4 (boilerplate): Synchronously wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // step5: Obtain the output value and copy the result from the device memory to the host side.
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), selfDeviceAddr,
                    size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
  aclDestroyTensor(self);
  aclDestroyTensor(batch1);
  aclDestroyTensor(batch2);
  aclDestroyScalar(alpha);
  aclDestroyScalar(beta);
  aclDestroyTensor(out);

  // 7. Release device resources. Modify based on the specific API definition.
  aclrtFree(selfDeviceAddr);
  aclrtFree(batch1DeviceAddr);
  aclrtFree(batch2DeviceAddr);
  aclrtFree(outDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();

  return 0;
}
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details