For Beginners

Function

This sample shows how to use the Runtime API of CANN and the Add operator in the operator library to implement the vector addition operation out = self + alpha * other.

Input vectors:
  self:   [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]
  other:  [0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0]
  alpha:  1.0

Vector addition result:
  out:   [1.5, 3.0, 4.5, 6.0, 7.5, 9.0, 10.5, 12.0]

App Build and Run

You can click Link to obtain the complete sample code. The README.md file of the sample also provides the compilation and running guide.

Understand key code logic.

This section describes the logic of the sample code in the application development sequence: Initialize resources → Create a stream → Prepare inputs → Call the Add operator → Output results → Release resources → Deinitialize resources. This section also helps you understand the functions of key APIs in CANN application development. The name prefix of these key APIs is acl, which is referred to as acl APIs in the following sections.

1. Initialize resources.

   1 2 3 4 5 6

   // Initialize the system. nullptr indicates that the default configuration is used to initialize the system. Before calling other ACL APIs, you must initialize the system. Otherwise, errors may occur during the initialization of internal system resources, causing exceptions of other services. aclInit(nullptr); // Specify the compute device. int32_t deviceId = 0; aclrtSetDevice(deviceId);

2. Create a stream..

   1 2 3

   // A stream is equivalent to a task queue. Tasks are executed in sequence based on the sequence in which they enter the queue. aclrtStream stream = nullptr; aclrtCreateStream(&stream);

3. Prepare for input.
   According to the calculation formula (out = self + alpha * other) of the Add operator, self and other are the two input vectors to be added, alpha is the coefficient, and out is the result vector after addition. Therefore, when preparing the input, you need to create two input tensors, a scalar representing coefficients, and an output tensor. When creating an input or output tensor, you need to allocate device memory to store the input or output data.

   1. Defines a common API for creating a tensor.

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

      template <typename T> int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType, aclTensor** tensor) { auto size = GetShapeSize(shape) * sizeof(T); // Allocate the device memory. aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST); // Copy the input data from the host to the device synchronously. aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE); // Calculate strides. std::vector<int64_t> strides(shape.size(), 1); for (int64_t i = shape.size() - 2; i >= 0; i--) { strides[i] = shape[i + 1] * strides[i + 1]; } // Create a tensor. *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(), *deviceAddr); return 0; }

   2. Create an input tensor, an output tensor, and a coefficient scalar.

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

      The vector length of std::vector<int64_t> shape{8}; // is 8. void* selfDeviceAddr = nullptr; void* otherDeviceAddr = nullptr; void* outDeviceAddr = nullptr; aclDataBuffer* outDataBuffer = nullptr; aclTensor* self = nullptr; aclTensor* other = nullptr; aclTensor* out = nullptr; aclScalar* alpha = nullptr; Input data of the std::vector<float> selfHostData = {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f}; // self vector Input data of the std::vector<float> otherHostData = {0.5f, 1.0f, 1.5f, 2.0f, 2.5f, 3.0f, 3.5f, 4.0f}; // other vector std::vector<float> outHostData(8, 0.0f); float alphaValue = 1.0f; // Coefficient value // Create an input tensor. CreateAclTensor(selfHostData, shape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self); CreateAclTensor(otherHostData, shape, &otherDeviceAddr, aclDataType::ACL_FLOAT, &other); // Create an alpha scalar. alpha = aclCreateScalar(&alphaValue, aclDataType::ACL_FLOAT); // Create an output tensor. CreateAclTensor(outHostData, shape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);

4. Call the Add operator..

   // Call the built-in CANN operator. Generally, the two-phase API is called.
   
   // The first-phase API aclnnAddGetWorkspaceSize is used to verify input parameters, deduce the output shape in the dynamic shape scenario, tile data, and calculate the workspace memory size required for operator execution.
   uint64_t workspaceSize = 0;
   aclOpExecutor* executor = nullptr;
   ret = aclnnAddGetWorkspaceSize(self, other, alpha, out, &workspaceSize, &executor);
   
   // The second-phase API aclnnAdd is used to perform operator computation. The API involves DFX (such as dump and overflow detection) and calls the LaunchKernel API provided by Runtime.
   // workspaceAddr indicates the address of the workspace memory allocated on the device.
   ret = aclnnAdd(workspaceAddr, workspaceSize, executor, stream);
   
   // Wait until the event is complete.
   aclrtSynchronizeStream(stream);

5. Output result.
   Returns the result data from the device to the host.

   1 2 3

   // outBufferAddr indicates the memory address for storing the result data on the device. // resultData.data () indicates the memory address for storing host-side data. aclrtMemcpy(resultData.data(), resultData.size() * sizeof(float), outBufferAddr, size * sizeof(float), ACL_MEMCPY_DEVICE_TO_HOST);

6. Release resources.

   // Destroy the tensor and scalar.
   aclDestroyTensor(self);
   aclDestroyTensor(other);
   aclDestroyScalar(alpha);
   aclDestroyTensor(out);
    
   // Release the device memory.
   aclrtFree(selfDeviceAddr);
   aclrtFree(otherDeviceAddr);
   aclrtFree(outDeviceAddr);
   aclrtFree(workspaceAddr);
   
   // Destroy the stream.
   aclrtDestroyStream(stream);

7. Deinitializes resources.

   1 2 3 4

   // Reset the device. aclrtResetDeviceForce(deviceId); // Deinitialization aclFinalize();