Runtime Resource Allocation and Deallocation

This section describes the runtime resources and how to allocate and deallocate these resources, and provides sample code.

The app you develop must contain the code logic for runtime resource allocation. See API Call Sequence to learn about the overall API call sequence before viewing the resource allocation and deallocation workflows and sample code in this section.

Principles

Allocate runtime resources, such as devices and streams, in sequence to support the execution of computing and management tasks. After all data is processed, deallocate these runtime resources in sequence.

Streams can be created implicitly or explicitly based on the application scenarios.

Implicit stream creation: applies to simple apps with low complicity of interaction logic. However, in multithreaded programming, each thread uses the default stream. The execution sequence of tasks in the default stream depends on the thread scheduling sequence of the operating system.
(Recommended) Explicit stream creation: applies to large apps with complex interaction logic, offering better app readability and maintainability.

Single-process, single-thread, and single-stream scenarios:

Single-process: One app maps to one process.
Single-thread: Only one thread is available by default unless you create more.
Single-stream: A single stream is used in the entire development process.
Asynchronous tasks in the same stream are executed in the sequence of tasks in the app.

The multithreading, multistreaming scenarios are described in Stream Management.

Runtime Resource Allocation

Figure 1 Runtime resource allocation workflow

Allocate the devices and streams in sequence.

Call aclrtSetDevice to specify the compute device. (You can also configure the default compute device during initialization. For details, see aclInit.) aclrtSetDevice also implicitly creates the default context and stream. However:
- A device corresponds to a default context that is released when aclrtResetDevice or aclrtResetDeviceForce is called. Therefore, you do not need to release the default context by explicit call.
- A device corresponds to a default stream that is released when aclrtResetDevice or aclrtResetDeviceForce is called. Therefore, you do not need to release the default stream by explicit call.
To use the default stream as the input parameter, pass NULL. Alternatively, you can call aclrtCreateStream to explicitly create a stream instead of using the default stream.
(Optional) Call aclrtGetRunMode to obtain the run mode of the software stack and determine the logic of the memory allocation API call based on the run mode.
If the query result is ACL_HOST, host memory needs to be allocated for data transfer.

If the query result is ACL_DEVICE, only device memory needs to be allocated for data transfer.

For details, see Data Transfer.

Runtime Resource Deallocation

Figure 2 Runtime resource deallocation workflow

Deallocate runtime resources in the following sequence: streams and devices.

If aclrtCreateStream is called to explicitly create a stream, aclrtDestroyStream needs to be called to destroy the stream. Otherwise, aclrtDestroyStream does not need to be called.
If you call aclrtResetDevice to destroy allocations on the device, this API will also implicitly destroy the default context and stream. You do not need to destroy the default context and stream separately.

Sample Code

You can view the complete code in Image Classification with ResNet-50 (Synchronous Inference).

Following the API calls, add exception handling branches and specify log printing of error and information levels. The following is a code snippet of key steps only, which is not ready to be built or run.

      
           // Initialize variables.
int32_t deviceId=0 ;
aclrtStream stream;
extern bool g_isDevice;

// =====Runtime resource allocation=====
//Specify a compute device.
aclError ret = aclrtSetDevice(deviceId);

//Explicitly create a stream.
//This is to reserve the execution order of asynchronous tasks.
ret = aclrtCreateStream(&stream);

//Obtain the run mode of the AI software stack. The subsequent API calls vary according to the run mode.
aclrtRunMode runMode;
ret = aclrtGetRunMode(&runMode);
g_isDevice = (runMode == ACL_DEVICE);
// =====Runtime resource allocation=====

// ......

// =====Runtime resource deallocation=====
ret = aclrtDestroyStream(stream);
ret = aclrtResetDevice(deviceId);
// =====Runtime resource deallocation=====

// ......

Parent topic: Runtime Management