Documentation
Download

mskpp.tiling_func

Function

Calls the tiling function of the user.

mskpp.tiling_func does not support the "GetCompileInfo" in Basic Data Structures and API Reference.

Function Prototype

def tiling_func(op_type: str, inputs: list, outputs: list, lib_path: str,
                inputs_info: list = None, outputs_info: list = None, attr=None, soc_version: str = None) -> TilingOutput

Parameters

Parameter

Input/Output

Description

op_type

Input

Set this parameter based on the implementation of the tiling function, for example, AddCustom and MatmulLeakyreluCustom. Unique basis for msKPP to search for the tiling function. For details about the search logic, see lib_path.

Data type: str.

This parameter is required.

NOTE:

If an operator of the same type (op_type) has been deployed in CANN, and you have modified the tiling function and recompiled the operator, you need to redeploy the operator in the CANN environment.

inputs

Input

Enter the tensor information in the sequence of the input parameters of the kernel function. If a parameter is not used, pass None in the corresponding position as a placeholder.

Data type: list. Each element must be a tensor or list[tensor]. If format or ori_format is not explicitly specified in inputs_info, all tensors are in ND format by default.

This parameter is optional.

outputs

Input

Enter the tensor information in the sequence of the input parameters of the kernel function. If a parameter is not used, pass None in the corresponding position as a placeholder.

Data type: list. Each element must be a tensor or list[tensor]. If format or ori_format is not explicitly specified in inputs_info, all tensors are in ND format by default.

This parameter is optional.

inputs_info

Input

Enter the information in the sequence of the input parameters of the kernel function. If a parameter is not used, pass None in the corresponding position as a placeholder.

Data type: list. The data type of the element in the inputs_info parameter is dict or list[dict]. The elements of each dict are described as follows:

  • ori_shape: original dimension information of the input tensor.
  • shape: dimension information of the input tensor during running.
  • dtype: data type of the input tensor. For details, see "DataType" in TBE&AI CPU Operator Development API.
  • ori_format: original data layout format of the input tensor. The default value is ND. For details, see "Format" in TBE&AI CPU Operator Development API.
  • format: data layout format of the input tensor. The default value is ND. For details, see "Format" in TBE&AI CPU Operator Development API.
  • data_path: path of the .bin file of the input tensor in the value dependency scenario.

The following is an example:

[{"ori_shape": [8, 2048], "shape": [8, 2048], "dtype": "float16", "ori_format": "ND", "format": "ND"},
 {"ori_shape": [8, 2048], "shape": [8, 2048], "dtype": "float16", "ori_format": "ND", "format": "ND"}]

This parameter is optional.

NOTE:

The input parameter has the following restrictions on inputs:

  • If inputs is a tensor, inputs_info must be dict.
  • If inputs is list[tensor], inputs_info must be list[dict].
  • If inputs is None, each element of inputs_info must contain at least [shape, dtype].

outputs_info

Input

Stores the output information. If a parameter is not used, pass an empty dict placeholder to the corresponding position.

Data type: list. The data type of the outputs_info element is dict or list[dict]. The elements of each dict are described as follows:

  • ori_shape: original dimension information of the output tensor.

The following is an example:

[{"shape": [8, 2048], "dtype": "float16", "format": "ND"},
 {"shape": [8, 2048], "dtype": "float16", "format": "ND"}]

This parameter is optional.

NOTE:

The input parameter has the following restrictions on inputs:

  • If outputs is a tensor, outputs_info must be dict.
  • If outputs is list[tensor], outputs_info must be list[dict].
  • If outputs is None, each element of outputs_info must contain at least [shape, dtype].

attr

Input

Operator attributes used by the tiling function .

Data type: dict or list.

NOTE:
  • The key value in dict format can contain only letters, digits, and underscores (_).
    {
  "a1": 1,
  "a2": False,
  "a3": "ssss",
  "a4": 1.2,
  "a5": [111, 222, 333],
  "a6": [111.111, 111.222, 111.333],
  "a7": [True, False],
  "a8": ["asdf", "zxcv"],
  "a9": [[1, 2, 3, 4], [5, 6, 7, 8], [5646, 2345]],
}
  • The list format is recommended. If an empty list needs to be passed for an attr, this format must be used (for example, "a10" in the following).
    • The values of "name" and "value" can contain only letters, digits, and underscores (_).
    • "dtype": data type of the input tensor.
    [
  {"name": "a1", "dtype": "int", "value": 1},
  {"name": "a2", "dtype": "bool", "value": False},
  {"name": "a3", "dtype": "str", "value": "ssss"},
  {"name": "a4", "dtype": "float", "value": 1.2},
  {"name": "a5", "dtype": "list_float", "value": [111.111, 111.222, 111.333]},
  {"name": "a6", "dtype": "list_bool", "value": [True, False]},
  {"name": "a7", "dtype": "list_str", "value": ["asdf", "zxcv"]},
  {"name": "a8", "dtype": "list_list_int", "value": [[1, 2, 3, 4], [5, 6, 7, 8], [5646, 2345]]},
  {"name": "a9", "dtype": "list_int", "value": [111, 222, 333]},
  {"name": "a10", "dtype": "list_int", "value": []},
  {"name": "a11", "dtype": "int64", "value": 2},
  {"name": "a12", "dtype": "float32", "value": 1.3},
  {"name": "a13", "dtype": "string", "value": "ssss"},
  {"name": "a14", "dtype": "list_string", "value": ["asdf", "zxcv"]},
]

This parameter is optional.

lib_path

Input

Path of the liboptiling.so file generated through msOpGen project compilation. You can run the find . -name 'liboptiling.so' command in the project directory to search for the file. msKPP obtains the tiling function of the user based on the search sequence of the deployed operator and .so file.

Data type: str.

This parameter is optional.

soc_version

Input

Sets Ascend AI Processor version.

This parameter is optional.

NOTE:
  • Products except for Atlas A3 training products/Atlas A3 inference products: Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy. If Ascendxxxyy is the code sample path, you need to set ascendxxxyy.
  • Atlas A3 training products/Atlas A3 inference products: Run the npu-smi info -t board -i id -c chip_id command on the server where the Ascend AI Processor is installed to obtain Chip Name and NPU Name. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. If Ascendxxx_1234 is the code sample path, you need to set ascendxxx_1234.

    Note that:

    • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
    • chip_id: chip ID, which is obtained by running the npu-smi info -m command.

Return Value

Parameter

Description

blockdim

Number of cores configured for the tiling function.

Data type: int.

workspace_size

The value is the size of the workspace applied for by the user plus 78,643,200 bytes reserved by msKPP.

Data type: int.

workspace

Workspace applied for by msKPP. The size is workspace_size.

Data type: numpy.array.

tiling_data

Stores tiling_data, which is used to call the kernel function.

Data type: numpy.array.

tiling_key

tiling_key configured by the user's tiling function. If it is not set, msKPP sets it to 0 by default.

Data type: int.

Example

M = 1024
N = 640
K = 256
input_a = np.random.randint(1, 10, [M, K]).astype(np.float16)
input_b = np.random.randint(1, 10, [K, N]).astype(np.float16)
input_bias = np.random.randint(1, 10, [N]).astype(np.float32)
output = np.zeros([M, N]).astype(np.float32)
# tiling data
tiling_output = mskpp.tiling_func(
    op_type="MatmulLeakyreluCustom",
    inputs=[input_a, input_b, input_bias], outputs=[output],
    lib_path="liboptiling.so",  # tiling code build output.
)
Parent topic: API List
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details