Documentation

Runtime Resource Allocation and Deallocation

This section describes the runtime resources and how to allocate and destroy these resources, and provides sample code.

The app you develop must contain the code logic for runtime resource allocation. See AscendCL API Call Sequence to learn about the overall API call sequence before viewing the resource allocation and deallocation workflows and sample code in this section.

Principles

Allocate runtime resources including the device and stream in sequence to support the execution of computing and management tasks. After all data is processed, deallocate runtime resources including the streams and devices in sequence.

Streams can be created implicitly or explicitly based on the application scenarios.
  • Implicit stream creation: applies to simple apps with low complicity of interaction logic. However, in multithreaded programming, each thread uses the default stream. The execution sequence of tasks in the default stream depends on the thread scheduling sequence of the operating system.
  • (Recommended) Explicit stream creation: applies to large apps with complex interaction logic, offering better app readability and maintainability.
Single-process, single-thread, and single-stream scenarios:
  • Single-process: One app maps to one process.
  • Single-thread: Only one thread is available by default unless you create more.
  • Single-stream: A single stream is used in the entire development process.

    For asynchronous tasks in the same stream, AscendCL executes the tasks in order.

Runtime Resource Allocation

Figure 1 Runtime resource allocation workflow
Allocate the devices and streams in sequence.
  1. Call aclrtSetDevice to specify the compute device. (You can also configure the default compute device during AscendCL initialization. For details, see aclInit.) aclrtSetDevice also implicitly creates the default context and stream. However:
    • One device corresponds to one default context. The default context is released when the aclrtResetDevice API is called to release resources.
    • One device corresponds to one default stream. The default stream does not need to be released by explicitly calling the API. It is released when the aclrtResetDevice API is called to release resources.
  2. To use the default stream as the input parameter, pass NULL. Alternatively, you can call aclrtCreateStream to explicitly create a stream instead of using the default stream.
  3. (Optional) Call aclrtGetRunMode to obtain the run mode of the software stack and determine the logic of the memory allocation API call based on the run mode.

    If the query result is ACL_HOST, host memory needs to be allocated for data transfer.

    If the query result is ACL_DEVICE, only device memory needs to be allocated for data transfer.

    For details, see Data Transfer.

Runtime Resource Deallocation

Figure 2 Runtime resource deallocation workflow
Deallocate runtime resources in the following sequence: stream and device.
  1. If the aclrtCreateStream API is called to explicitly create a stream, the aclrtDestroyStream API needs to be called to release the stream. Otherwise, the aclrtDestroyStream API does not need to be called.
  2. If you call aclrtResetDevice to destroy allocations on the device. This API also implicitly destroys the default context and stream. You do not need to destroy the default context and stream separately.

Sample Code

You can view the complete code in Image Classification with ResNet-50 (Synchronous Inference).

After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
 
// Initialize variables.
int32_t deviceId=0 ;
aclrtStream stream;
extern bool g_isDevice;

// =====Runtime resource allocation=====
// Specify a compute device.
aclError ret = aclrtSetDevice(deviceId);

// Explicitly create a stream.
// To reserve the execution order of asynchronous tasks.
ret = aclrtCreateStream(&stream);

// Obtain the run mode of the AI software stack. The subsequent API calling varies according to the run mode.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
g_isDevice = (runMode == ACL_DEVICE);
// =====Runtime resource allocation=====

// ......

// =====Runtime resource deallocation=====
ret = aclrtDestroyStream(stream);
ret = aclrtResetDevice(deviceId);
// =====Runtime resource deallocation=====

// ......
Parent topic: Runtime Management