The msKPP tool provides two types of APIs: basic APIs and instruction APIs. Basic APIs simulate the chip platform and basic data for operator computing. Instruction APIs simulate specific operator instruction operations, including Vector and Cube computing instructions.
Table 1 msKPP API list
Function
Processor abstraction, which is instantiated and used in the [object Object] statement to explicitly model a type of the Ascend AI Processor.
Prototype
Parameter Description
Member Description
Constraints
This class needs to be initialized under the [object Object] statement.
Example
[object Object]
Returns
None
Function
AI Core abstraction, which is instantiated and used in the [object Object] statement to model an AI Core type.
Prototype
Parameter Description
Constraints
This class needs to be initialized under the [object Object] statement.
Example
Returns
None
Function
Onboard tensor abstraction, in which the memory location, data type, size, and format of tensors can be specified as the data dependency identifiers of instructions.
Prototype
Parameter Description
Member Description
Constraints
You need to create a tensor whose shape is [object Object] and [object Object] for scalar creation.
Example
Returns
None
Function
All data transfer instructions in the msKPP tool are abstracted as the load method, and users only need to focus on the reasonable transfer channels in Ascend AI Processors, without considering the complex stride concept in the transfer instructions.
Prototype
Parameter Description
Constraints
[object Object] and [object Object] must be used in pairs. Otherwise, pipeline blocking may occur.
The [object Object] parameter supports only the following four transfer channels: L1_TO_L0A, L1_TO_L0B, GM_TO_L0A, and GM_TO_L0B.
Returns
None
Function
Ensures the synchronization of different instructions between pipelines in a core. After [object Object] is scheduled, [object Object] is unblocked. After [object Object] and [object Object] are set, the will better meet the user's expectations.
Prototype
Parameter Description
Constraints
- The number of
[object Object]instructions must match the number of[object Object]instructions in the same core. - Duplicate
[object Object]instructions should not exist in the same core. - In the same core, if the values of
[object Object]and[object Object]in[object Object]and[object Object]are the same,[object Object]must be unique.
Example
Returns
None
Function
Ensures the synchronization of different instructions across pipelines in a core. [object Object] is unblocked after [object Object] is scheduled.
Prototype
Parameter Description
Constraints
- The number of
[object Object]instructions must match the number of[object Object]instructions in the same core. - Duplicate
[object Object]instructions should not exist in the same core. - In the same core, if the values of
[object Object]and[object Object]in[object Object]and[object Object]are the same,[object Object]must be unique.
Example
Returns
None
Function
Performs matrix multiplication and addition.
Prototype
Parameter Description
Constraints
When the bias term is in the bias table space, the tensor data format must be ND and shape must be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are added by element.
Prototype
Parameter Description
Constraints
The tensors of all input and output data of vector instructions are in the UB space, and their shapes must be the same.
Example
Returns
None
Function
[object Object] instruction abstraction.
Expands the dimensions of the tensors based on the instruction stride. However, the msKPP instruction system does not support stride. Therefore, you need to specify the dimension expansion factor and ensure that the shapes of the input and output tensors are the same.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] indicates the vector calculation for type conversion of input data.
Currently, the following type conversion is supported: BF16->FP32, FP16->FP32, FP16->INT16, FP16->INT32, FP16->INT4, FP16->INT8, FP16->UINT8, FP32->BF16, FP32->FP16, FP32->INT32, FP32->INT64, INT4->FP16, INT64->FP32, INT8->FP16, and UINT8->FP16.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] indicates vector calculation for quantization on input data.
Currently, conversions from FP16 to INT8 and from INT32 to FP16 are supported.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] indicates vector calculation for quantization on input data.
Currently, the conversion from INT16 to INT8 is supported.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are filled in by element.
Prototype
Parameter Description
Constraints
Since the input to this instruction is only a scalar, you need to create a tensor whose shape is [object Object] and [object Object] as the simulated scalar input, without increasing performance overhead.
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] take exponents by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] take logarithms by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] take the maximum value by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are multiplied by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] evaluates the product of vector [object Object] and scalar [object Object].
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] is subtracted by [object Object] by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] is divided by [object Object] by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Reduces the tensor dimensions based on the input parameters of the instruction. In the msKPP instruction system, [object Object] controls the shape reduction multiple and ensures that the shapes of the input and output tensors are the same. When the last dimension of the shape is reduced to [object Object], the dimension is eliminated. Ensure that the last dimension of the shape can be exactly divided by [object Object] and is not [object Object].
Prototype
Parameter Description
Example
Constraints
The value of [object Object] cannot be [object Object].
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] take the absolute value by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are added by element before the relu value is calculated.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are added by element, the relu value is calculated, and the output is quantized.
The following conversion types are supported: FP16->INT8, FP32->FP16, and INT16->INT8.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] evaluates the sum of vector [object Object] and scalar [object Object].
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] can be obtained when [object Object] and [object Object] perform AND operation by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object]. vaxpy calculates the product of vector [object Object] and scalar [object Object], and adds the target address [object Object]. The output data type can be specified as FP32 by using [object Object].
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Sorts data based on the x input and provides the original index data of the elements after sorting. Therefore, the shape of the output vector tensor is twice that of the x data.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the sum of elements in each block. There are eight blocks in total. Mixed addresses are not supported.
Prototype
Parameter Description
Constraints
The value of [object Object] cannot be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the maximum element of each block. There are eight blocks in total. Mixed addresses are not supported.
Prototype
Parameter Description
Constraints
The value of [object Object] cannot be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the minimum element of each block. There are eight blocks in total. Mixed addresses are not supported.
Prototype
Parameter Description
Constraints
The value of [object Object] cannot be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the maximum element value in the input vector.
Prototype
Parameter Description
Constraints
The value of [object Object] cannot be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the minimum element value in the input vector.
Prototype
Parameter Description
Constraints
The value of [object Object] cannot be [object Object].
Example
Returns
None
Function
[object Object] instruction abstraction. The following six instructions have the same performance.
[object Object], where [object Object] can be obtained when [object Object] is equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than or equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is less than or equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is smaller than [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is not equal to [object Object] by element-wise comparison.
Prototype
Parameter Description
Constraints
The tensors of all input and output data of vector instructions are in the UB space, and their shapes must be the same.
Example
Returns
None
Function
[object Object] instruction abstraction. The following six instructions have the same performance.
[object Object], where [object Object] can be obtained when [object Object] is equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than or equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is less than or equal to [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is smaller than [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is not equal to [object Object] by element-wise comparison.
Prototype
Parameter Description
Constraints
The tensors of all input and output data of vector instructions are in the UB space, and their shapes must be the same.
Example
Returns
None
Function
[object Object] instruction abstraction. The following six instructions have the same performance.
[object Object], where [object Object] can be obtained when [object Object] is equal to scalar stored in [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than or equal to scalar stored in [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is greater than scalar stored in [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is less than or equal to scalar stored in [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is smaller than scalar stored in [object Object] by element-wise comparison.
[object Object], where [object Object] can be obtained when [object Object] is not equal to scalar stored in [object Object] by element-wise comparison.
Prototype
Parameter Description
Constraints
The tensors of all input and output data of vector instructions are in the UB space, and their shapes must be the same.
Example
Returns
None
Function
[object Object] instruction abstraction.
Copies tensors at the source address to the destination address.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the sum of n and n+1 of the input [object Object] vector, and writes the result back to y. n is an even index. [object Object] controls the output type.
Prototype
Parameter Description
Example
Returns
None
Function
Gathers given input tensors by element to the result tensor based on the offset address tensor provided.
Prototype
Parameter Description
Example
Returns
None
Function
Gathers a given input tensor to the result tensor based on the offset address tensor provided.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
If [object Object] is greater than or equal to 0, [object Object] = [object Object]. If [object Object] is less than 0, [object Object], where [object Object] is multiplied by scalar [object Object] by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object] = [object Object] × [object Object] + [object Object]. Performs multiplication and addition on each element of the two vectors.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object]: Performs multiplication and addition on each element of the two vectors, and then performs an MADDRELU operation on each element in the result.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Compares each element in the vector with a scalar and returns the larger one.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Compares each element in two vectors with a scalar and returns the smaller one.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Compares each element in the vector with a scalar and returns the smaller one.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are multiplied by element, and the multiplication result is added to [object Object] by element. The output data type can be specified as FP32 by using [object Object].
The value can be:
type = f16: f16 = f16 × f16 + f16
type = f32: f32 = f32 × f32 + f32
[object Object]: f32 = f16 × f16 + f32. The [object Object] and [object Object] vectors use 64-element f16 data for calculation. The source vector uses only the lower four blocks, and the upper four blocks are ignored. [object Object] is 64-element f32 data with eight blocks, and is used as both the target vector and the third source vector.
Prototype
Parameter Description
Constraints
The tensors of input and output data of vector instructions are in the UB space.
Example
Returns
None
Function
Merges at most four sorted Region Proposal lists into one. The results are sorted in descending order of the [object Object] fields.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are multiplied by element, and the output is quantized.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Performs bitwise NOT on input vectors. Each vector has 8 × 256 bits.
Prototype
Parameter Description
Example
Constraints
This instruction supports only the common mask mode and counter mode.
Returns
None
Function
[object Object] instruction abstraction.
Performs bitwise OR on input vectors. Each vector has 8 × 256 bits.
Prototype
Parameter Description
Example
Constraints
This instruction supports only the common mask mode and counter mode.
Returns
None
Function
[object Object] instruction abstraction.
Performs floating-point reciprocal estimation and finds an approximate reciprocal for each vector.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Determines which elements of the [object Object] vector are to be stored in the [object Object] vector based on the mask data of the input [object Object] vector. Because the tensor in msKPP lacks actual elements, the [object Object] parameter is added to determine the shape of the [object Object] output.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Determines which block-level elements of the [object Object] vector are to be stored in the [object Object] vector based on the mask data of the input [object Object] vector. Because the tensor in msKPP lacks related concepts, the [object Object] parameter is added to determine the shape of the [object Object] output.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Performs the relu operation on each element, which takes 0 if the element is less than 0, and takes the element itself if it is greater than or equal to 0.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Calculates the reciprocal square root of a floating point number.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
This function is usually used in conjunction with [object Object], which selects an element in the corresponding positions of [object Object] and [object Object] based on the obtained [object Object].
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Performs logical left shift or arithmetic left shift based on the input type.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Performs logical right shift or arithmetic right shift based on the input type.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], which takes the square root of [object Object] by element.
Prototype
Parameter Description
Example
Constraints
The input value must be a positive number. Otherwise, the result is unknown and an exception occurs.
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where the relu value is calculated after [object Object] is subtracted by [object Object] by element.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
[object Object], where [object Object] and [object Object] are subtracted by element, the relu value is calculated, and the output is quantized.
The following conversion types are supported: FP16->INT8, FP32->FP16, and INT16->INT8.
Prototype
Parameter Description
Example
Returns
None
Function
[object Object] instruction abstraction.
Transposes a 16 × 16 matrix starting from the input address [object Object] (32-byte aligned). Each element has 16 bits. The result is output to [object Object]. The input and output are continuous 512-byte storage spaces.
Prototype
Parameter Description
Example
Returns
None