Matrix Multiplication with TSCM Inputs

Overview

TSCM represents the logical memory corresponding to L1 Buffer. For details about L1 Buffer, see the storage units. You can manage TSCM to efficiently utilize hardware resources. For example, you can cache a copy of TSCM data and flexibly configure it as matrix A, matrix B, or a bias matrix for Matmul operations in different scenarios, thereby implementing memory reuse and optimizing computational efficiency. In the TSCM input scenario, you manage the entire TSCM memory space. Matmul directly uses the passed-in TSCM memory address and does not move data from the global memory to TSCM.

Application Scenarios

You need to customize data movement to TSCM and customize management. In other words, you need to customize the data movement function, such as non-contiguous movement or preprocessing of the moved-in data. By customizing TSCM management, you can flexibly configure the MTE2 pipeline to implement global DoubleBuffer across Matmul objects. For details about MTE2, see the DMA units.

Restrictions

The matrices configured as TSCM inputs must be fully loaded to TSCM, meaning all matrix data is moved to at once and stored in TSCM.

Example

For complete operator examples, see Matmul operator sample with custom GM data source for TSCM input and BatchMatmul operator sample with custom TSCM input.

      
       
         
         
           TQue<TPosition::A1, 1> scm; // The logical location of the queue is A1, and the queue depth is 1.
pipe->InitBuffer(scm, 1, tiling.M * tiling.Ka * sizeof(A_T)); 
// The TPosition of A_TYPE is TSCM, while the TPosition of B_TYPE is GM.
Matmul<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE> mm1;
REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm1);
mm1.Init(&tiling);
// Customize matrix A movement from GM to TSCM.
auto scmTensor = scm.AllocTensor<A_T>();
DataCopy(scmTensor, gm_a, tiling.M * tiling.Ka);
scm.EnQue(scmTensor);
LocalTensor<A_T> scmLocal = scm.DeQue<A_T>();
// Set matrix A for TSCM inputs and matrix B for GM inputs.
mm1.SetTensorA(scmLocal);
mm1.SetTensorB(gm_b);
mm1.SetBias(gm_bias);
mm1.IterateAll(gm_c);
scm.FreeTensor(scmLocal);

          

        

      
     

Parent topic: Feature Scenarios