Overview

In eager mode (adopted by mainstream frameworks such as PyTorch), each operation or task is launched and executed on the fly, without building a computational graph. This mode provides immediately visible execution results and convenient debugging capabilities, but it also introduces host-side dispatch overhead. As the performance optimization continues, the host overheads gradually become the bottleneck and an issue that cannot be ignored.

On the AI processor, you can offload related tasks to the device for execution, which can reduce the host overheads. To achieve this, an acl API is provided for capturing stream tasks to the model and then executing the model (which is called ACL Graph for short). Between aclmdlRICaptureBegin and aclmdlRICaptureEnd, all tasks dispatched to the specified stream are not executed immediately. Instead, they are temporarily stored in the running instance of the model. These tasks are executed only when aclmdlRIExecuteAsync is called to execute the model. If the tasks in a stream need to be executed multiple times, you do not need to dispatch the tasks again. Instead, you only need to call aclmdlRIExecuteAsync to execute the model multiple times. This reduces the task dispatch overhead on the host. After the tasks are executed, if the running instance of the model is no longer required, you can call aclmdlRIDestroy to destroy the resource in a timely manner.

Note: This function is for trial use and may be changed in later versions. It is not available in production environments.