Architecture Changes from 220x to 351x

Figure 1 shows the 351x architecture. Compared with 220x, the 351x architecture has the following new features:

• Added multiple data channels.
• Increased the number of AI Cores.
• Increased the UB capacity.
• Added the SSBuffer intra-core storage unit, which can be accessed by the AIC and AIV cores through the scalar unit.
• Supported SIMT programming and SIMD and SIMT programming in addition to SIMD programming.
• Adopted the RegBase architecture for AIV cores. Compared with the MemBase architecture of 220x, the RegBase architecture allows direct operations on the vector registers of the chip, achieving greater flexibility and better performance.

The following tables describe the major changes in the 351x architecture. In addition, the 351x architecture supports more data types. For details, see Applicability.

• DMA unit

  Table 1 DMA unit changes

  Change in 351x	Impact	Affected API
  Deleted the data channel from the L1 buffer to the global memory.	The existing APIs do not support direct data movement from the L1 buffer to the global memory. You need to allocate a space in the L1 buffer to store the identity matrix, use the MMAD matrix multiplication to compute the data and output it to the L0C buffer, and then use FixPipe to move the data from the L0C buffer to the global memory.	DataCopy/DumpTensor
  Deleted the data channels from the global memory to the L0A buffer and L0B buffer.	The original data movement from the global memory to the L0A buffer and L0B buffer needs to be split into two steps: data movement from the global memory to the L1 buffer and data movement from the L1 buffer to the L0A buffer and L0B buffer.	LoadData
  Added a data channel from the unified buffer to the L1 buffer.	Data can be directly moved from the unified buffer to the L1 buffer, without being moved from the unified buffer to the global memory and then from the global memory to the L1 buffer. For details, see Basic Data Movement.	DataCopy
  Added ND-DMA instructions.	Capabilities of the DataCopy (data movement) API are extended. Compared with the basic data movement API, extended capabilities of the API allow for more flexible configuration of the dimension information and stride of the data to be moved. For details, see Multi-dimensional Data Transfer (ISASI).	DataCopy
  Added a unidirectional data channel from the L0C buffer to the unified buffer.	Data can be directly moved from the L0C buffer to the unified buffer, without being moved from the L0C buffer to the global memory and then from the global memory to the unified buffer. For details, see Fixpipe.	Fixpipe
  Extended the LoadData movement instruction.	Data movement in the MicroScaling (MX) scenario is supported. For details, see LoadData.	LoadData
  Added the DN fractal and removed support for transpose in movement from the L1 buffer to the L0A buffer.	For details, see LoadDataWithTranspose.	LoadDataWithTranspose
  Added NZ2DN channel conversion to FixPipe (to implement channel conversion of the data format from NZ to DN).	For details, see Fixpipe.	Fixpipe
  Enhanced the DataCopy movement dimension.	DataCopy supports loop-mode movement between the L1 buffer and global memory and between the global memory and unified buffer. For details, see SetLoopModePara.	DataCopy
  Deleted hardware instructions related to the initialization of the L0A buffer and L0B buffer.	The basic API InitConstValue is used to initialize the LocalTensor at a specific storage location to a specific value. The LocalTensor in the L0A buffer and L0B buffer cannot be directly initialized.	InitConstValue

• Compute unit

  Table 2 Compute unit changes

  Change in 351x	Impact	Affected API
  The cube unit does not support the s4 type.	For matrix multiplication of the int4b_t data type, you need to cast the int4b_t data to the int8_t data before performing cube computation.	Mmad
  The cube unit does not support the fractal change from ZZ to ZN in the L0A buffer.	In the L0A buffer tiling scenario, the L0A buffer address of the left matrix needs to be recomputed for matrix multiplication.	LoadData/LoadDataWithTranspose
  The vector core MemBase architecture is switched to the RegBase architecture.	The performance of basic APIs deteriorates in some scenarios.	Basic APIs in high-dimensional sharding mode
  The hardware does not support the subnormal function. Currently, the subnormal function is implemented through software simulation.	You need to set the config template parameters to configure the subnormal computation mode. For details, see Vector Computation.	Ln/Sqrt/Rsqrt/Div/Reciprocal/Exp
  4:2 sparse matrix computation is not supported.	You need to use the Vector Core capability to convert a dense matrix into a sparse matrix.	LoadDataWithSparse/MmadWithSparse

• Storage unit

  Table 3 Storage unit changes

  Change in 351x	Impact	Affected API
  Deleted the boundary value setting of the L1 buffer.	In the 351x architecture hardware, the registers related to the boundary value setting of the L1 buffer are deleted and the SetLoadDataBoundary API is no longer supported. For details, see Compatibility Solution.	SetLoadDataBoundary
  Changed the unified buffer structure. For details about the comparison between the unified buffer structures of the 220x and 351x architectures, see the bank structure comparison.	In the 220x architecture, the unified buffer is divided into 16 bank groups. Each bank group contains three banks, and size of each bank is 4 KB. In the 351x architecture, the unified buffer is divided into 8 bank groups. Each bank group contains two banks, and size of each bank is 16 KB. If a unified buffer conflict occurs, you can refer to Preventing Bank Conflicts of the Unified Buffer to resolve the conflict.	/

• Synchronization

  Table 4 Synchronization changes

  Change in 351x	Impact	Affected API
  Added Mutex capabilities.	Mutex is used for synchronization between asynchronous pipeline instructions in a core. Its function is similar to the lock mechanism in traditional CPUs. The synchronization dependency between pipelines is implemented by locking specified pipelines and then releasing them. For details, see Mutex (ISASI).	Mutex
  Added the inter-core synchronization control mode.	For internal synchronization control of the AI Core, AIV0 and AIV1 can separately trigger AIC waiting. For details, see CrossCoreSetFlag (ISASI).	CrossCoreSetFlag/CrossCoreWaitFlag

• Others

  Table 5 Other changes

  Change in 351x	Impact	Affected API
  Deleted the AIPP hardware-level instructions and implemented the AIPP function through software simulation.	The performance of the AIPP API may deteriorate.	SetAippFunctions/LoadImageToLocal
  Deleted the unified buffer exception debugging API because related registers in the 351x architecture are deleted.	The debugging API has no impact on functions.	CheckLocalMemoryIA