ReduceSum

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	√
Atlas A2 training product / Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	√

Function Usage

Sums up all input data.

ReduceSum can be implemented in either of the following ways:

Method 1: Binary tree accumulation is first performed on a repeat and then performed on the results of different repeats.
Assume that the source operand is 128 data elements of the half type [data0, data1, data2, ..., data127], the computation can be completed in one repeat. The computation process is as follows:
1. Add data0 and data1 to obtain data00, add data2 and data3 to obtain data01, ..., add data124 and data125 to obtain data62, and add data126 and data127 to obtain data63.
2. Add data00 and data01 to obtain data000, add data02 and data03 to obtain data001, ..., and add data62 and data63 to obtain data031.
3. By analogy, the destination operand is one data element of the half type ([data]).
When being greater than 65504, the computation result is truncated to 65504. For example, the source operand is [60000, 60000, –30000, 100], 60000 + 60000 > 65504, meaning that the result overflows. In this case, the maximum value 65504 will be used as the result. Similarly, –30000 + 100 = –29900, 65504 – 29900 = 35604.
Method 2: Binary tree accumulation is performed on a repeat, and results of different repeats are accumulated in sequence.

The following table lists the ReduceSum methods for different hardware forms.

For the Atlas A3 training product / Atlas A3 inference product , method 2 is used for computing the first n data elements of a tensor, and method 1 is used for the high-dimensional tensor sharding computation.

For the Atlas A2 training product / Atlas A2 inference product , method 2 is used for computing the first n data elements of a tensor, and method 1 is used for the high-dimensional tensor sharding computation.

For the Atlas 200I/500 A2 inference product , method 1 is used.

For the Atlas inference product AI Core, method 1 is used.

For the Atlas training product , method 1 is used.

Prototype

Computation of the first n data elements of a tensor

        
             template <typename T, bool isSetMask = true>
__aicore__ inline void ReduceSum(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LocalTensor<T>& sharedTmpBuffer, const int32_t count)

High-dimensional tensor sharding computation

Bitwise mask mode

          
               template <typename T>
__aicore__ inline void ReduceSum(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LocalTensor<T>& sharedTmpBuffer, const uint64_t mask[], const int32_t repeatTime, const int32_t srcRepStride)

Contiguous mask mode

          
               template <typename T>
__aicore__ inline void ReduceSum(const LocalTensor<T>& dst, const LocalTensor<T>& src, const LocalTensor<T>& sharedTmpBuffer, const int32_t mask, const int32_t repeatTime, const int32_t srcRepStride)

Parameters

**Table 1** Parameters in the template
Parameter	Description
T	Operand data type. For the Atlas 350 Accelerator Card, the supported data types are half, float, uint64_t, and int64_t. For the Atlas A3 training product / Atlas A3 inference product , the supported data types are half and float. For the Atlas A2 training product / Atlas A2 inference product , the supported data types are half and float. For the Atlas 200I/500 A2 inference product , the supported data types are half and float. For the Atlas inference product AI Core, the supported data types are half and float. For the Atlas training product , the supported data type is half.
isSetMask	Whether to set mask inside the API. The default value is true. true: sets mask inside the API. false: sets mask outside the API. Developers need to use the SetVectorMask API to set the mask value. In this mode, the mask value in the input parameter of this API must be set to the placeholder MASK_PLACEHOLDER.

**Table 2** Parameters
Parameter	Input/Output	Meaning
dst	Output	Destination operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. The start address of the LocalTensor must be 2-byte aligned (for data of the half type) or 4-byte aligned (for data of the float type).
src	Input	Source operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. The start address of LocalTensor must be 32-byte aligned. The source operand must have the same data type as the destination operand.
sharedTmpBuffer	Input	Buffer used for storing intermediate results during instruction execution and serving as the workspace required for internal calculations. For details about the computation method, see sharedTmpBuffer size computation. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. The start address of LocalTensor must be 32-byte aligned. Its data type must match that of the destination operand.
count	Input	Number of elements involved in the computation. The parameter value range is related to the operand data type. The maximum number of elements that can be processed varies according to the data type. However, the maximum size of data that can be processed cannot exceed the UB size limit.
mask/mask[]	Input	mask controls the elements that participate in computation in each iteration. Bitwise mode: controls the elements that participate in computation by bit. If a bit is set to 1, the corresponding element participates in the computation. If a bit is set to 0, the corresponding element is masked from the computation. The mask value is an array. The array length and the value range of the array elements are related to the operand data type. When the operand is 16-bit, the array length is 2, with mask[0] and mask[1] each in the range [0, 2⁶⁴ – 1], and they cannot both be 0 at the same time. When the operand is 32-bit, the array length is 1, with mask[0] in the range (0, 2⁶⁴ – 1]. When the operand is 64-bit, the array length is 1, with mask[0] in the range (0, 2³² – 1]. For example, if mask = [0, 8] and 8 = 0b1000, only the fourth element participates in computation. Contiguous mode: indicates the number of contiguous elements that participate in computation. The value range is related to the operand data type. The maximum number of elements that can be processed in each iteration varies according to the data type. When the operand is 16-bit, mask ∈ [1, 128]. When the operand is 32-bit, mask ∈ [1, 64]. When the operand is 64-bit, mask ∈ [1, 32].
repeatTime	Input	Number of iteration repeats. Unlike High-dimensional Sharding APIs, a larger value range is supported. Ensure that the value does not exceed the maximum value of int32_t.
srcRepStride	Input	Address stride between adjacent iterations of the source operand, that is, the number of data blocks skipped from the source operand in each iteration. For details, see repeatStride.

For the two accumulation methods described in the function usage, the size of sharedTmpBuffer is computed as follows:

Method 1: Compute the minimum space required according to the following formula:

          
               // Define a round-up function.
int RoundUp(int a, int b)
{ 
    return (a + b - 1) / b;
}

// Define the data types involved in the computation.
int typeSize = 2;                           // half occupies 2 bytes, and float occupies 4 bytes. Set this parameter as required.

// Define two units based on the data type.
int elementsPerBlock = 32 / typeSize;       // Number of elements that a data block can hold
int elementsPerRepeat = 256 / typeSize;     // Number of elements that can be processed in a repeat

// Determine the first maximum repeat value.
int firstMaxRepeat = repeatTime;           // For high-dimensional tensor sharding computation APIs, firstMaxRepeat is repeatTime. For APIs of computing the first n data elements of a tensor, firstMaxRepeat is count/elementsPerRepeat. For example, firstMaxRepeat is count/128 for the half type and count/64 for the float type. Set this parameter as required. For count<elementsPerRepeat, the value of firstMaxRepeat is 1.

int iter1OutputCount = firstMaxRepeat;                                              // Number of elements generated in the first repeat
int iter1AlignEnd = RoundUp(iter1OutputCount, elementsPerBlock) * elementsPerBlock; // Round up the number of elements generated in the first repeat.
int finalWorkLocalNeedSize = iter1AlignEnd;                                         // The final size of sharedTmpBuffer (in elements) equals the ceiling value of elements generated in the first computation round.

Method 2: Pass a sharedTmpBuffer of any size. The value of sharedTmpBuffer will not be changed.

Returns

None

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
For details about the restrictions on operand address overlapping, see General Address Overlapping Restrictions. If sharedTmpBuffer is required, address overlapping between dst and sharedTmpBuffer is supported (usually dst requires less space than sharedTmpBuffer). However, sharedTmpBuffer must meet the minimum space requirement; otherwise, address overlapping is not supported.

For the Atlas 350 Accelerator Card, uint64_t and int64_t support only the API for computing the first n data elements of the tensor.
The ReduceSum API is implemented through software simulation. In some scenarios, the performance of this API may be lower than that of the BlockReduceSum and WholeReduceSum APIs implemented by using hardware instructions. Proper use of the reduction instruction in different scenarios can improve performance. For details about the introduction, see Selecting Low-Latency Instructions to Optimize Reduction Operation Performance. For details about examples, see ReduceCustom.

Examples

Example of high-dimensional tensor sharding computation (contiguous mask mode)

        
             // dstLocal, srcLocal, and sharedTmpBuffer are of the half type. The computation data volume of srcLocal is set to 8320. The data is continuously arranged. The high-dimensional tensor sharding computation is used. repeatTimes is set to 65. mask indicates that all elements are involved in the computation.
int32_t mask = 128;
AscendC::ReduceSum<half>(dstLocal, srcLocal, sharedTmpBuffer, mask, 65, 8);

Example of high-dimensional tensor sharding computation (bitwise mask mode)

        
             // dstLocal, srcLocal, and sharedTmpBuffer are of the half type. The computation data volume of srcLocal is set to 8320. The data is continuously arranged. The high-dimensional tensor sharding computation is used. repeatTimes is set to 65. mask indicates that all elements are involved in the computation.
uint64_t mask[2] = { 0xFFFFFFFFFFFFFFFF, 0xFFFFFFFFFFFFFFFF };
AscendC::ReduceSum<half>(dstLocal, srcLocal, sharedTmpBuffer, mask, 65, 8);

Example of computing the first n data elements of a tensor

        
             // dstLocal, srcLocal, and sharedTmpBuffer are of the half type. For srcLocal, the computation data volume is 8320 and is continuously arranged. It uses the computation API for the first n tensor elements.
AscendC::ReduceSum<half>(dstLocal, srcLocal, sharedTmpBuffer, 8320);

The following is a complete example of the high-dimensional tensor sharding computation API:

        
             #include "kernel_operator.h"
int srcDataSize = 8320;
int dstDataSize = 16;
int mask = 128;
int repStride = 8;
int repeat = srcDataSize / mask; // 65

// Initialize srcLocal, dstLocal, and sharedTmpBuffer.
AscendC::LocalTensor<half> srcLocal = inQueueSrc.DeQue<half>();
AscendC::LocalTensor<half> dstLocal = outQueueDst.AllocTensor<half>();
AscendC::LocalTensor<half> sharedTmpBuffer = workQueue.AllocTensor<half>();
// If mask has a value of 128, 128 elements are computed at once. The computation of 8320 numbers is achieved through 65 repeats.
AscendC::ReduceSum<half>(dstLocal, srcLocal, sharedTmpBuffer, mask, repeat, repStride);
// Release the tensor.
outQueueDst.EnQue<half>(dstLocal);
inQueueSrc.FreeTensor(srcLocal);
workQueue.FreeTensor(sharedTmpBuffer);

The following is an example:

Input (src_gm):
[1. 1. 1. ... 1. 1. 1.]
Output (dst_gm):
[8320.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.
    0.    0.    0.    0.]

The following is a complete example of the computation API for the first n data elements of a tensor:

        
             #include "kernel_operator.h"

int srcDataSize = 288;
// Initialize srcLocal, dstLocal, and sharedTmpBuffer.
AscendC::LocalTensor<half> srcLocal = inQueueSrc.DeQue<half>();
AscendC::LocalTensor<half> dstLocal = outQueueDst.AllocTensor<half>();
AscendC::LocalTensor<half> sharedTmpBuffer = workQueue.AllocTensor<half>();

// Compute the first 288 numbers using the level 2 API and compute the sum of the first 288 numbers.
AscendC::ReduceSum<half>(dstLocal, srcLocal, sharedTmpBuffer, srcDataSize);
// Release the tensor.
outQueueDst.EnQue<half>(dstLocal);
inQueueSrc.FreeTensor(srcLocal);
workQueue.FreeTensor(sharedTmpBuffer);

The following is an example:

Input (src_gm):
[1. 1. 1. ... 1. 1. 1.]
Output (dst_gm):
[288.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.]

Parent topic: Reduction Computation