aclnnAddmv

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	√
Atlas training products	√

Function

Description: Performs matrix-vector multiplication and then adds the result to a vector.
Formula:
$out = β self + α (mat @ vec)$

Function Prototype

Each operator is divided into a two-phase API. The "aclnnAddmvGetWorkspaceSize" API must be called first to obtain the workspace size required for computation and the executor that encapsulates the operator computation flow. Then, the "aclnnAddmv" API is called to perform the computation.

aclnnStatus aclnnAddmvGetWorkspaceSize(
  const aclTensor* self,
  const aclTensor* mat,
  const aclTensor* vec,
  const aclScalar* alpha,
  const aclScalar* beta,
  aclTensor*       out,
  int8_t           cubeMathType,
  uint64_t*        workspaceSize,
  aclOpExecutor**  executor)

aclnnStatus aclnnAddmv(
  void*           workspace,
  uint64_t        workspaceSize,
  aclOpExecutor*  executor,
  aclrtStream     stream)

aclnnAddmvGetWorkspaceSize

Parameters

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Shape	Non-contiguous Tensor
self	Input	A 1D vector to be added to the subsequent multiplication result.	The data type must form a deduction relationship with mat@vec. When alpha is not 0, the shape must satisfy the broadcast relationship with mat@vec. When alpha is 0, the shape must be the same as mat@vec.	BFLOAT16, FLOAT16, FLOAT, INT32, INT64, INT16, INT8, UINT8, DOUBLE, BOOL	ND	1	√
mat	Input	A 2D matrix that is multiplied with vec.	The data type must have a deduction relationship with self. The shape must satisfy a multiplication relationship with vec.	BFLOAT16, FLOAT16, FLOAT, INT32, INT64, INT16, INT8, UINT8, DOUBLE, BOOL	ND	2	√
vec	Input	A 1D vector that performs multiplication with mat.	The data type needs to have a deduction relationship with self. The shape needs to satisfy the multiplication relationship with mat.	BFLOAT16, FLOAT16, FLOAT, INT32, INT64, INT16, INT8, UINT8, DOUBLE, BOOL	ND	1	√
alpha	Input	Coefficient of the product of mat and vec in the formula.	-	FLOAT、FLOAT16、DOUBLE、INT8、INT16、INT32、INT64、UINT8	-	-	-
beta	Input	Coefficient of self in the formula.	-	FLOAT, FLOAT16, DOUBLE, INT8, INT16, INT32, INT64, UINT8	-	-	-
out	Output	Specifies the 1D output vector.	The data type must be the deduced data type of self, mat, vec, alpha, and beta. The shape is the same as the product of mat and vec.	BFLOAT16, FLOAT16, FLOAT, INT32, INT64, INT16, INT8, UINT8, DOUBLE	ND	1	√
cubeMathType	Input	Specifies the computation logic of the Cube unit.	If the input data types have a deduction relationship, this parameter processes the data type after deduction by default. The supported enumeration values are as follows: 0: KEEP_DTYPE, keeps the input data type for computation. 1: ALLOW_FP32_DOWN_PRECISION, allows reducing the precision of input data for computation. 2: USE_FP16, allows reducing the input precision to FLOAT16 for computation. 3: USE_HF32, allows reducing the input precision to HFLOAT32 for computation.	INT8	-	-	-
workspaceSize	Output	Returns the workspace size to be requested on the Device side.	-	-	-	-	-
executor	Output Parameter	Returns the operator executor, which contains the operator computation process.	-	-	-	-	-

Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products, Ascend 950PR/Ascend 950DT:
- cubeMathType=1: When the input data type is FLOAT32, it is converted to HFLOAT32 for computation. No processing is performed for other input data types.
- cubeMathType=2: This option is not supported when the input data type is BFLOAT16.
- cubeMathType=3: When the input data type is FLOAT32, it is converted to HFLOAT32 for computation. This option is not supported for other input data types.
Atlas training products, Atlas inference products:
- BFLOAT16 data type is not supported.
- cubeMathType=0 is not supported when the input data type is FLOAT32.
- For cubeMathType=1, when the input data type is FLOAT32, the computation is converted to FLOAT16; when the input is of other data types, no conversion is performed.
- cubeMathType=3 is not supported.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Value	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input self, mat, vec, alpha, beta, or out is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The data type or data format of self, mat, or vec is not within the supported range.
		The data type cannot be deduced from self, mat, and vec.
		The deduced data type/data format cannot be converted to the type/format of the specified output out.
		The shapes of mat and vec do not meet the multiplication condition, or self and the multiplication result do not meet the addition condition.

aclnnAddmv

Parameters

Parameter	Input/Output	Description
workspace	Input	Specifies the workspace memory address applied on the Device side.
workspaceSize	Input	Specifies the workspace size applied on the Device side, which is obtained through the first-phase API aclnnAddmvGetWorkspaceSize.
executor	Input	Operator executor that contains the operator computation process.
stream	Input	Specifies the stream that executes the task.

Return Value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation description:
- Atlas training products and Atlas inference products: aclnnAddmv defaults to a non-deterministic implementation. Deterministic computation can be enabled by calling aclrtCtxSetSysParamOpt.
- Ascend 950PR/Ascend 950DT: aclnnAddmv defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_addmv.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate. Initialize resources.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for device memory.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Call aclrtMemcpy to copy data from the host to the device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of a contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call aclCreateTensor to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
  // Fill in the deviceId based on your actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API interface.
  std::vector<int64_t> selfShape = {2};
  std::vector<int64_t> matShape = {2, 2};
  std::vector<int64_t> vecShape = {2};
  std::vector<int64_t> outShape = {2};
  void* selfDeviceAddr = nullptr;
  void* matDeviceAddr = nullptr;
  void* vecDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  aclTensor* self = nullptr;
  aclTensor* mat = nullptr;
  aclTensor* vec = nullptr;
  aclScalar* alpha = nullptr;
  aclScalar* beta = nullptr;
  aclTensor* out = nullptr;
  std::vector<float> selfHostData = {1, 1};
  std::vector<float> matHostData = {1, 1, 1, 1};
  std::vector<float> vecHostData = {1, 1};
  std::vector<float> outHostData(2, 0);
  int8_t cubeMathType = 1;
  float alphaValue = 1.0f;
  float betaValue = 1.0f;
  // Create the self aclTensor.
  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the mat aclTensor.
  ret = CreateAclTensor(matHostData, matShape, &matDeviceAddr, aclDataType::ACL_FLOAT, &mat);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the vec aclTensor.
  ret = CreateAclTensor(vecHostData, vecShape, &vecDeviceAddr, aclDataType::ACL_FLOAT, &vec);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  // Create the alpha aclScalar.
  alpha = aclCreateScalar(&alphaValue,aclDataType::ACL_FLOAT);
  CHECK_RET(alpha != nullptr, return ret);
  // Create the beta aclScalar.
  beta = aclCreateScalar(&betaValue,aclDataType::ACL_FLOAT);
  CHECK_RET(beta != nullptr, return ret);
  // Create the out aclTensor.
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Replace with the actual API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnAddmv.
  ret = aclnnAddmvGetWorkspaceSize(self, mat, vec, alpha, beta, out, cubeMathType, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddmvGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);

  // Apply for device memory based on the workspaceSize computed by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnAddmv.
  ret = aclnnAddmv(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAddmv failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronously wait for the task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.
  auto size = GetShapeSize(outShape);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr,
                    size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  // 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
  aclDestroyTensor(self);
  aclDestroyTensor(mat);
  aclDestroyTensor(vec);
  aclDestroyScalar(alpha);
  aclDestroyScalar(beta);
  aclDestroyTensor(out);

  // 7. Release device resources. Modify based on the specific API definition.
  aclrtFree(selfDeviceAddr);
  aclrtFree(matDeviceAddr);
  aclrtFree(vecDeviceAddr);
  aclrtFree(outDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();

  return 0;
}