ACL Graph Overview

In eager mode (adopted by mainstream frameworks such as PyTorch), each operation or task is delivered and executed at the same time, and there is no need to build a computational graph. This mode not only brings about immediate execution and convenient debugging, but also results in the delivery overhead of the host. As the performance optimization continues, the host overheads gradually become the bottleneck and an issue that cannot be ignored.

On the Ascend AI Processor, you can offload related tasks to the device for execution, which can reduce the host overheads. To achieve this, an acl API is provided for capturing stream tasks to the model and then executing the model (which is called ACL Graph for short). Between aclmdlRICaptureBegin and aclmdlRICaptureEnd, all tasks delivered to the specified stream are not executed immediately. Instead, they are temporarily stored in the running instance of the model. These tasks are executed only when aclmdlRIExecuteAsync is called to execute the model. If the tasks in a stream need to be executed multiple times, you do not need to deliver the tasks again. Instead, you only need to call aclmdlRIExecuteAsync to execute the model multiple times. This reduces the task delivery overhead on the host. After the tasks are executed, if the running instance of the model is no longer required, you can call aclmdlRIDestroy to destroy the resource in a timely manner.

Note: This feature is for trial use and may be changed in later versions. It is not available in commercial products.