Basics

This section describes how to use the Reg vector compute API and SIMT API for SIMD and SIMT programming.

In Vector Core, the SIMT unit and SIMD unit share the on-chip memory. Therefore, the on-chip memory Unified Buffer can be used to complete SIMD and SIMT programming. For details about the hardware architecture, see NPU Architecture Version 351x. Before learning the following content, you need to understand the SIMD and SIMT programming model: SIMD and SIMT Programming.

SIMD programming provides register-based (Regbase) Reg vector compute APIs. These APIs can directly operate the SIMD registers in the Vector Core. The maximum amount of data that can be processed by an API at a time is equal to the size of the register, which can be obtained by calling GetVecLen. In operator implementation, you need to call the Reg vector compute API for multiple times to process single-core data.

Different from SIMD programming, in SIMT programming, data on the Global Memory can be directly read and used. During SIMT programming, data is tiled by organizing thread hierarchies. SIMT Built-In Keywords such as threadIdx are used to compute the data index to be processed by a thread and complete the compute of data corresponding to the index. In this way, the function implementation is simplified to scalar compute.