Constraints
- For the following computation types, the input is broadcast before the specified operation is performed: ELEWISE_ADD/ELEWISE_MUL/ELEWISE_REALDIV/ELEWISE_SUB/ELEWISE_LOGICAL_AND/ELEWISE_LOGICAL_OR/ELEWISE_LESS/ELEWISE_GREATER/ELEWISE_EQUAL
- The values of the dimensions of input x and y must be the same, or at least one of them must be 1.
- ELEWISE_DEQUANT_PER_CHANNEL supports only
Atlas A2 training products /Atlas A2 inference products andAtlas A3 inference products /Atlas A3 training products . - For
Atlas inference products , ELEWISE_QUANT_PER_CHANNEL supports only 32-bit aligned data. - For the input x of the ELEWISE_DYNAMIC_QUANT calculation type, the restrictions are as follows:
- For
Atlas A2 training products /Atlas A2 inference products , the maximum data amount supported by the tail axis of the float16 type is 26624, and the maximum data amount supported by the tail axis of the bf16 type is 7552. Currently, only symmetric quantization is supported. - For
Atlas inference products , the tail axis supports only 32-bit aligned data, and the maximum data amount supported by the tail axis is 4096. Currently, only symmetric quantization is supported. - On
Atlas A2 training products /Atlas A2 inference products andAtlas A3 inference products /Atlas A3 training products , outTensorType can be empty or 2. - On Atlas 350 accelerator card, outTensorType can be empty, 2, 34, 35, or 36.
- For
- ELEWISE_QUANT constraints:
- Core constraint formula:
- Definition of key parameters
Parameter
Physical Meaning
Remarks
numRow
Product of all dimensions except the last dimension (that is,
).Total number of data rows.
numCol
Size of the last dimension (that is,
).Data row length.
maxUbsize
Unified buffer (UB) size.
Fixed value 196608.
maxCoreNum
Maximum number of available hardware cores.
See the following table.
Product Series
maxCoreNum
Atlas A3 inference products /Atlas A3 training products 48
Atlas A2 training products /Atlas A2 inference products 40
- Calculation process of firstDimPertimes (number of rows transferred or processed at a time)
- Calculating the number of rows allocated to a single core
Calculate the number of actually launched cores and the number of rows allocated to each core based on the total number of rows and the maximum number of hardware cores.
- Calculating the thresholds of the required memory and available memory
- Determining the number of rows to be split
Determine the value of firstDimPertimes based on the memory sufficiency.
- Calculating the number of rows allocated to a single core
Parent topic: ElewiseOperation