Introduction to TSCM

Tasks communicate and synchronize with each other through queues between the Vector and Cube. TSCM is a data structure used to manage and perform queue-related operations and allocate related resources when the destination of the data path is the TSCM position. TSCM, TQueBind, and TQue are of the same type structure. The definition of TSCM is as follows:

     
          template <TPosition pos, int32_t depth, auto mask = 0>
using TSCM = TQueBind<pos, TPosition::TSCM, depth, mask>;

Table 1 Template parameters

Parameter

Meaning

pos

Logical location of a queue. It can be VECIN or GM. For details about TPosition, see TPosition.

depth

The depth of a queue indicates the number of consecutive enqueue or dequeue operations that can be performed in the queue. During code running, if there are n consecutive enqueue operations (with no dequeue operations in between) in the queue, its depth needs to be set to n.

Note that the queue depth is irrelevant to double buffering. The queue mechanism is used to implement pipeline parallelism. On this basis, double buffering further improves the pipeline utilization. Even if the queue depth is 1, double buffering can still be enabled.

When the queue depth is set to 1, the compiler is specially optimized for better performance. The recommended value is 1.

In the following example, the queue is not enqueued consecutively, and the queue depth is set to 1.

            
                 a1 = que.AllocTensor(); 
que.EnQue(a1);
a1 = que.DeQue();
que.FreeTensor(a1);

In the following example, the queue is enqueued twice consecutively, and the queue depth should be set to 2. This setting is used only in a few preload scenarios. (For example, two pieces of data are moved in consecutively. After one piece is computed, another piece is moved in, and then the other piece previously moved in is computed.) In other cases, a depth greater than or equal to 2 is not recommended.

            
                 a1 = que.AllocTensor(); 
a2 = que.AllocTensor();
que.EnQue(a1);
que.EnQue(a2);
a1 = que.DeQue();
a2 = que.DeQue(); 
que.FreeTensor(a1);
que.FreeTensor(a2);

mask

If mask is of the int type, bits are used to express information.
- Bit 0 is a reserved parameter.
- Bit 1 is a reserved parameter.
- If bit 2 is 0, the memory locations of VECTOR0 and VECTOR1 that are mapped to the Cube are staggered. If bit 2 is 1, the memory locations of VECTOR0 and VECTOR1 that are mapped to the Cube are the same.

If mask is of the const TQueConfig* type, the structure and parameters of TQueConfig are defined as follows.

            
                 struct TQueConfig {
    bool nd2nz = false;  // Not supported. The default value is false.
    bool nz2nd = false;  // Not supported. The default value is false.
    bool scmBlockGroup = false;  // TSCM-related parameter. If this parameter is set to false, the memory locations of VECTOR0 and VECTOR1 that are mapped to the Cube are staggered. If this parameter is set to true, the memory locations of VECTOR0 and VECTOR1 that are mapped to the Cube are the same.
    uint32_t bufferLen = 0;  // The value must be the same as the value of len entered when InitBuffer is used. Performance can be optimized during compilation. The value 0 indicates that resources are allocated when InitBuffer is used.
    uint32_t bufferNumber = 0;  // The value must be the same as the value of num entered when InitBuffer is used. Performance can be optimized during compilation. The value 0 indicates that resources are allocated when InitBuffer is used.
    uint32_t consumerSize = 0;  // Reserved
    TPosition consumer[8] = {}; // Reserved
    bool enableStaticEvtId = false; // Reserved
    bool enableLoopQueue = false;   // Reserved
};

TSCM is defined via using as an alias for TQueBind when the destination address is specified as the TSCM position.
- The AllocTensor, EnQue, DeQue, and FreeTensor APIs are supported. The operations of the APIs must be performed in the sequence of AllocTensor->EnQue->DeQue->FreeTensor, and the APIs must be used in pairs.
- However, TSCM does not need to support all APIs of TQueBind. VacantInQue, HasTensorInQue, GetTensorCountInQue, and HasIdleBuffer are not supported.
The buffer allocated by TSCM stores synchronization event IDs, and the structure is used together with Cube high-level APIs (such as Matmul called by the macro function). Therefore, the number of TSCM buffers on the same TPosition is related to the numbers of hardware synchronization events IDs and Matmul objects.
The maximum total number of buffers initiated by TSCM from VECIN and Matmul objects is 10.

The requested TSCM buffer cannot exceed the specification limit. Otherwise, undefined behaviors may occur.

The following is a simple example:

     
          TSCM<TPosition::VECIN, 1> tscm;
for () {
    auto scmTensor = tscm.AllocTensor<float>(); // Allocate the buffer before data is moved from UB to TSCM.
    DataCopy(scmTensor, ubLocal, 1024); // Move the UB data to TSCM for matmul computation.
    tscm.EnQue(scmTensor); // After data movement, perform enqueue and dequeue operations before matmul computation.
    LocalTensor<float> scmLocal = tscm.DeQue<float>();
    mm.SetTensorA(scmLocal);
    mm.SetTensorB(gm_b);
    mm.IterateAll(gm_c);
    tscm.FreeTensor(scmLocal); // Release the tensor after the matmul computation is completed.
}

The following is an example of calling this API together with Matmul:

     
      
        
        
          {
    typedef matmul::MatmulType<AscendC::TPosition::TSCM, CubeFormat::NZ, half, true, LayoutMode::NONE, false, AscendC::TPosition::VECIN> A_TYPE;
    typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> B_TYPE;
    typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> C_TYPE;
    typedef matmul::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> BIAS_TYPE; 
    matmul::Matmul<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE> mm1;
    constexpr uint32_t M = 32;
    constexpr uint32_t N = 32;
    constexpr uint32_t K = 32;
    AscendC::GlobalTensor<half> aGlobal;
    AscendC::GlobalTensor<half> bGlobal;
    AscendC::GlobalTensor<float> cGlobal;
    AscendC::GlobalTensor<float> biasGlobal;
    aGlobal.SetGlobalBuffer(reinterpret_cast<__gm__ half *>(aGM), M * K);
    bGlobal.SetGlobalBuffer(reinterpret_cast<__gm__ half *>(bGM), K * N);
    cGlobal.SetGlobalBuffer(reinterpret_cast<__gm__ float *>(cGM), M * N);
    TCubeTiling tiling;
    AscendC::TPipe pipe;
    AscendC::TSCM<AscendC::TPosition::VECIN, 1> scm;
    AscendC::TQue<AscendC::TPosition::VECIN, 1> qIn;
    pipe.InitBuffer(scm, 1, M * K * sizeof(half));
    pipe.InitBuffer(qIn, 1, M * K * sizeof(half));
    REGIST_MATMUL_OBJ(&pipe, workspaceGM, mm1, &tiling);
    auto scmTensor = scm.AllocTensor<half>();
    auto ubTensor = qIn.AllocTensor<half>();
    AscendC::Nd2NzParams intriParams;
    DataCopy(ubTensor, aGlobal, M * K);
    qIn.EnQue(ubTensor);
    AscendC::LocalTensor<half> ubLocal = qIn.DeQue<half>();
    AscendC::DataCopy(scmTensor, ubLocal, intriParams);
    scm.EnQue(scmTensor);
    AscendC::LocalTensor<half> scmLocal = scm.DeQue<half>();
    mm1.SetTensorA(scmLocal);
    mm1.SetTensorB(bGlobal);
    mm1.IterateAll(cGlobal);
    scm.FreeTensor(scmLocal);
    qIn.FreeTensor(ubLocal);
}

         

       

     
    

Parent topic: TSCM