Programming Model Overview

An AI Core is the compute core of an AI processor. An AI processor usually integrates multiple AI Cores to implement parallel compute. Compared with traditional CPUs, AI processors are more suitable for model training and inference because they have more compute units and corresponding vector compute instructions. This enables a single hardware instruction to compute multiple groups of data. AI processors provide the following programming models:

• Single Instruction Multiple Data (SIMD): implements parallel compute by using a single instruction and multiple groups of data.
• Single Instruction Multiple Thread (SIMT): implements parallel compute by using a single instruction and multiple threads.

The AI CPU is a processor on the device. It has the same memory access capability as the AI Core and can directly access the memory resources on the device. In addition, the AI CPU can perform data compute like the CPU on the host.

• SIMD programming

  This programming model is suitable for vector operators and fused operators that involve matrix compute and continuous compute. In addition, the hybrid programming of SIMD and SIMT can fully utilize the advantages of both models to achieve better performance and higher efficiency. For details about SIMD programming and hybrid programming of SIMD and SIMT, see AI Core SIMD Programming. For details about the basic operator development process, see SIMD Operator Implementation.

• SIMT programming

  This programming model is suitable for discrete access scenarios and complex branch control scenarios of vector operators. It also helps personnel familiar with SIMT operator development quickly master operator development on the AI processor. Currently, only Atlas 350 Accelerator Card is supported. To further learn about SIMT programming, you can refer to AI Core SIMT Programming to understand the detailed SIMT programming principles and SIMT Operator Implementation to learn about the basic process of SIMT operator development.

• AI CPU programming

  This programming model is usually used as a supplement to the AI Core and is responsible for non-matrix and selection-intensive computation with complex logic. You can learn about the basic knowledge of the AI CPU programming model by referring to AI CPU Programming.