GatherMask

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Selects elements from the source operand and writes them to the destination operand based on a gather mask (for data collection) that corresponds to either the binary of the built-in fixed mode or the binary of the user-defined input tensor values.

Prototype

User-defined mode

template <typename T, typename U, GatherMaskMode mode = defaultGatherMaskMode>
__aicore__ inline void GatherMask(const LocalTensor<T>& dst, const LocalTensor<T>& src0, const LocalTensor<U>& src1Pattern, const bool reduceMode, const uint32_t mask, const GatherMaskParams& gatherMaskParams, uint64_t& rsvdCnt)

Built-in fixed mode

template <typename T, GatherMaskMode mode = defaultGatherMaskMode>
__aicore__ inline void GatherMask(const LocalTensor<T>& dst, const LocalTensor<T>& src0, const uint8_t src1Pattern, const bool reduceMode, const uint32_t mask, const GatherMaskParams& gatherMaskParams, uint64_t& rsvdCnt)

Parameters

**Table 1** Template parameters
Parameter	Meaning
T	Data type of the source operand src0 and destination operand dst. For the Atlas 350 Accelerator Card, the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, bfloat16_t, float, int32_t, and uint32_t. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half, bfloat16_t, uint16_t, int16_t, float, uint32_t, and int32_t. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half, bfloat16_t, uint16_t, int16_t, float, uint32_t, and int32_t. For the Atlas 200I/500 A2 inference product, the supported data types are half, uint16_t, int16_t, float, uint32_t, and int32_t. For the Atlas inference product AI Core, the supported data types are half, uint16_t, int16_t, float, uint32_t, and int32_t.
U	Data type of src1Pattern in user-defined mode. The supported data types are uint8_t/, uint16_t, and uint32_t. If the data type of the destination operand is uint8_t or int8_t, src1Pattern must be of the uint8_t type. For 16-bit destination operands, the data type of src1Pattern must be uint16_t. If the data type of the destination operand is float, uint32_t, or int32_t, src1Pattern must be of the uint32_t data type.
mode	Reserved. A default value is provided. You do not need to set this parameter.

**Table 2** Parameters
Parameter	Input/Output	Meaning
dst	Output	Destination operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. The start address of LocalTensor must be 32-byte aligned.
src0	Input	Source operand. The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT. The start address of LocalTensor must be 32-byte aligned. The data type must be the same as that of the destination operand.
src1Pattern	Input	There are two modes of gather mask (for data collection): built-in fixed mode and custom mode. Elements are from the source operand and written to the destination operand based on either the binary of the built-in fixed mode or the binary of the custom input tensor values. 1: selected; 0: not selected. Built-in fixed mode: The data type of src1Pattern is uint8_t, and the value range is [1,7]. All repeat iterations use the same gather mask. src1RepeatStride cannot be configured. 1: 01010101…0101 # Obtain an even number of index elements for each repeat. 2: 10101010…1010 # Obtain an odd number of index elements for each repeat. 3: 00010001…0001 # Obtain the first element among every four elements in each repeat. 4: 00100010…0010 # Obtain the second element among every four elements in each repeat. 5: 01000100…0100 # Obtain the third element among every four elements in each repeat. 6: 10001000…1000 # Obtain the fourth element among every four elements in each repeat. 7: 11111111...1111 # Obtain all elements in each repeat. The Atlas 350 Accelerator Card supports modes 1 to 7. The Atlas A3 training product/Atlas A3 inference product supports modes 1 to 7. The Atlas A2 training product/Atlas A2 inference product supports modes 1 to 7. The Atlas 200I/500 A2 inference product supports modes 1 to 7. The Atlas inference product AI Core supports modes 1 to 6. User-defined mode: The data type of src1Pattern is LocalTensor. The iteration interval is determined by src1RepeatStride. src1Pattern is continuously consumed in an iteration.
reduceMode	Input	This parameter is used to select the mode of the mask parameter, and it is of the bool type. The options are as follows: false: normal mode. In this mode, 256-byte data is operated in each repeat operation. The total data computation amount is as follows: repeatTimes x 256 bytes. The mask parameter is invalid. You are advised to set it to 0. Set repeatTimes, src0BlockStride, and src0RepeatStride as required. src1Pattern can be set to the built-in fixed mode or user-defined mode. In user-defined mode, src1RepeatStride can be set as required. true: counter mode. Based on the meanings of parameters such as mask, this mode can be configured in either of the following methods: Configuration method 1: mask elements are operated in each repeat operation. The total data computation amount is as follows: repeatTimes x Number of mask elements. The mask value is set to the number of elements computed in each repeat. Set repeatTimes, src0BlockStride, and src0RepeatStride as required. src1Pattern can be set to the built-in fixed mode or user-defined mode. In user-defined mode, src1RepeatStride can be set as required. Configuration method 2: The total data computation amount is the number of mask elements. The mask value is configured as the total data computation amount. The repeatTimes value does not take effect. The number of iterations of an instruction is determined by the source operand and mask. Set src0BlockStride and src0RepeatStride as required. src1Pattern can be set to the built-in fixed mode or user-defined mode. In user-defined mode, src1RepeatStride can be set as required. For the Atlas 350 Accelerator Card, configuration method 1 is supported. For the Atlas A3 training product/Atlas A3 inference product, configuration method 1 is supported. For the Atlas A2 training product/Atlas A2 inference product, configuration method 1 is supported. For the Atlas 200I/500 A2 inference product, configuration method 1 is supported. For the Atlas inference product AI Core, configuration method 2 is supported.
mask	Input	This parameter is used to control the elements that participate in computation in each iteration. Based on reduceMode, there are two mask modes: Normal mode: The mask parameter is invalid. You are advised to set it to 0. Counter mode: The value range is [1, 2³² – 1]. The meaning of the mask parameter in counter mode varies according to the product model. For details about the configuration methods, see the description of the reduceMode parameter.
gatherMaskParams	Input	Data structure that controls operand address strides. It is of GatherMaskParams type. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_gather.h. Replace ${INSTALL_DIR} with the installation directory of CANN. For details, see Table 3.
rsvdCnt	Output	Number of elements retained by the instruction, corresponding to the number of valid elements in dstLocal. The data type is uint64_t.

**Table 3** GatherMaskParams parameters
Parameter	Description
src0BlockStride	Address stride between different data blocks in an iteration of src0 (interval between start addresses). Unit: data block
repeatTimes	Number of repeats.
src0RepeatStride	Address stride between adjacent iterations of src0 (interval between start addresses). Unit: data block
src1RepeatStride	Address stride between adjacent iterations of src1 (interval between start addresses). Unit: data block

Returns

None

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
For details about the constraints on operand address overlapping, see General Address Overlapping Restrictions.

If the counter mode was used before this API is called, set the mode back to counter after this API is called. (The reason is that the counter mode switches to the normal mode after this API is called.)

Examples

In the following example, a user-defined tensor is used:

uint32_t mask = 70; // Elements involved in computation in each iteration
uint64_t rsvdCnt = 0; // Number of retained elements

// src0Local: source operand
// src1Local: tensor for storing the mask for data collection
// dstLocal: destination operand
// reduceMode = true; Counter mode is used.
// The parameters in {} are as follows:
// src0BlockStride = 1; Data is continuously read and written at an interval of one data block in a single iteration.
// repeatTimes = 2; In counter mode, this parameter takes effect only for certain product models.
// src0RepeatStride = 4; The interval between iterations of the source operand is four data blocks.
// src1RepeatStride = 0; The interval between iterations of src1 is 0 data blocks, that is, the data is read from the original position.
AscendC::GatherMask (dstLocal, src0Local, src1Local, true, mask, { 1, 2, 4, 0 }, rsvdCnt);

The following figure shows configuration method 1 in counter mode.

mask = 70. 70 elements are computed in each repeat.
repeatTimes = 2. There are two repeats in total.
src0BlockStride = 1. There is no interval between data blocks in a single iteration of the source operand src0Local.
src0RepeatStride = 4. The interval between adjacent iterations of the source operand src0Local is four data blocks. Therefore, the second repeat starts from the 33rd element.
src1Pattern is set to the user-defined mode. src1RepeatStride = 0. The interval between adjacent iterations of src1Pattern is 0 data blocks. Therefore, the second repeat still starts from the start address of src1Pattern.

Figure 1 Configuration method 1 in counter mode

The following figure shows configuration method 2 in counter mode.

mask = 70. A total of 70 elements are computed.
repeatTimes does not take effect. The number of repeats is automatically inferred based on the source operand and mask. The data type of the source operand is uint32_t. Each iteration processes 256 bytes of data (64 elements). So, two repeats are required.
src0BlockStride = 1. There is no interval between data blocks in a single iteration of the source operand src0Local.
src0RepeatStride = 4. The interval between adjacent iterations of the source operand src0Local is four data blocks. Therefore, the second repeat starts from the 33rd element.
src1Pattern is set to the user-defined mode. src1RepeatStride = 0. The interval between adjacent iterations of src1Pattern is 0 data blocks. Therefore, the second repeat still starts from the start address of src1Pattern.

Figure 2 Configuration method 2 in counter mode

In the following example, the built-in fixed mode is used:

uint32_t mask = 0; // Elements involved in computation in each iteration. In normal mode, you are advised to set mask to 0.
uint64_t rsvdCnt = 0; // Number of elements retained after filtering
uint8_t src1Pattern = 2; // Built-in fixed mode

// src0Local: source operand
// src1Pattern: tensor for storing the mask for data collection
// dstLocal: destination operand
// reduceMode = false; Normal mode is used.
// The parameters in {} are as follows:
// src0BlockStride = 1; Data is continuously read and written at an interval of one data block in a single iteration.
// repeatTimes = 1; One repeat
// src0RepeatStride = 0; There is only one repeat, so set src0RepeatStride to 0.
// src1RepeatStride = 0; There is only one iteration, so set src1RepeatStride to 0.
AscendC::GatherMask(dstLocal, src0Local, src1Pattern, false, mask, { 1, 1, 0, 0 }, rsvdCnt);

Result example:

Input (src0Local): [1 2 3 ... 128]
Input (src1Pattern): src1Pattern = 2;
Output (dstLocal): [2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100 102 104 106 108 110 112 114 116 118 120 122 124 126 128 undefined ..undefined]
Output (rsvdCnt): 64

Parent topic: Comparison and Selection