Heterogeneous Parallelism Programming Model
Host-Device Heterogeneous Collaboration Mechanism
The Ascend C heterogeneous parallel programming model is designed to address the challenges of heterogeneous computing architectures and aims to solve the efficiency and scalability issues of traditional programming models when handling complex compute tasks.
The heterogeneous compute architecture involves the host and device (AI processor on the device). They work together to complete compute tasks. The host is responsible for runtime management, including storage management, device management, and stream management, to ensure efficient task scheduling and proper resource allocation. The device executes the kernel function compiled based on the Ascend C syntax to perform compute-intensive tasks, such as matrix and vector operations on batch data, for compute acceleration.
As shown in the following figure, when a kernel is delivered to the AI Core (compute core on AI processor) for execution, the runtime management module starts the corresponding task based on the number of cores and task type set by the developer. The task is loaded from the host to the stream running queue on the device. The scheduling unit allocates the ready task to an idle AI Core for execution. In this case, the data to be processed is split and run on multiple compute cores at the same time to obtain higher performance.
The host and device have different memory spaces. The host cannot directly access the device memory, and vice versa. Therefore, the input data needs to be copied from the host to the device memory for computation on the device, and the output result needs to be copied back from the device memory to the host for further use.
