Generating Single-Operator Board Test Framework

This section describes how to specify the ST case definition file (.json) and implementation file kernel_name.cpp of the Ascend C operator to automatically generate the board test framework for calling the kernel function, test and verify the operator, and view the output result to check whether the operator function is correct.

This function supports only Atlas inference products and Atlas training products, but not Atlas A2 training products/Atlas A2 inference products and Atlas A3 Training Series Product.
The addr and tiling attributes cannot be specified for any parameter.
The #ifndef __CCE_KT_TEST__ can be used to encapsulate kernel function calls.

Prepare the following input files:
- ST case definition file (*.json) of the operator.
- Operator implementation file (*.cpp) on the kernel. For details, see Operator Implementation on the Kernel in the Ascend C Operator Development Guide.
Run the following command to generate the test code to call kernel function. For details about the parameters, see Table 3.
```
msopst ascendc_test -i xx/OpType_case.json -kernel xx/add_custom.cpp -out ./output_data
```

View the execution result.

After the command is executed, the message Process finished! is displayed. A timestamp directory is generated in the directory specified by -out. In the timestamp directory, a folder named after OpType of the operator is generated for storing test cases and test results. The directory structure is as follows:

 {time_stamp}
│   ├── OpType
│   │   ├── CMakeLists.txt     // Compilation script
│   │   ├── data    
│   │   │   └── xx.bin        
│   │   │   └── xx.bin
│   │   ├── data_utils.h
│   │   ├── main.cpp        // Test framework
│   │   └── run.sh      // Script file for invoking the test framework
│   └── st_report.json        // Execution report

After the command is executed successfully, an st_report.json report is automatically generated. The report records the detailed test information and the runtime states in each phase, which facilitates troubleshooting and test result comparison.

The st_report.json file is saved in The st_report saved in.

2024-01-17 08:40:55 (3271037) - [INFO] Create 1 sub test cases for Test_AddCustom_001.
2024-01-17 08:40:55 (3271037) - [INFO] [STEP2] [data_generator.py] Generate data for testcase.
2024-01-17 08:40:55 (3271037) - [INFO] Start to generate the input data for Test_AddCustom_001_case_001_ND_float.
2024-01-17 08:40:55 (3271037) - [INFO] Generate data for testcase in $HOME/AddCustom/output/20240117084055/AddCustom/data.
2024-01-17 08:40:55 (3271037) - [INFO] [STEP3] [gen_ascendc_test.py] Generate test code of calling of kernel function for AscendC operator.
2024-01-17 08:40:55 (3271037) - [INFO] Content appended to $HOME/AddCustom/output/20240117084055/AddCustom/main.cpp successfully.
2024-01-17 08:40:55 (3271037) - [INFO] AscendC operator test code files for kernel implement have been successfully generated.
2024-01-17 08:40:55 (3271037) - [INFO] If you want to execute kernel function in Ascend aihost or cpu, please execute commands: cd $HOME/AddCustom/output/20240117084055/AddCustom && bash run.sh [KERNEL_NAME](add_custom) [SOC_VERSION](ascendxxxyy) [CORE_TYPE](AiCore/VectorCore) [RUN_MODE](cpu/npu). For example: cd $HOME/AddCustom/output/20240117084055/AddCustom && bash run.sh add_custom ascendxxxyy AiCore npu
2024-01-17 08:40:55 (3271037) - [INFO] Process finished!
2024-01-17 08:40:55 (3271037) - [INFO] The st report saved in: $HOME/AddCustom/output/20240117084055/st_report.json.

**Table 1** st_report.json description
Field			Description
run_cmd	-	-	Command
report_list	-	-	List of reports of test cases.
	trace_detail	-	Execution details.
		st_case_info	Test information, including the following: expect_data_path: path of the expected result case_name: test case name input_data_path: path of input data planned_output_data_paths: path of the actual result op_params: operator parameter information
		stage_result	Result information in each runtime phase, including the following: status: running status of the phase, either success or failure result: output result stage_name: stage name cmd: command
	case_name	-	Test name
	status	-	Actual test result, either success or failure
	expect	-	Expected test result, either success or failure
summary	-	-	Summary of comparison of the actual test result and the expected test result.
	test case count	-	Number of test cases
	success count	-	Number of test cases whose result is the same as the expected result
	failed count	-	Number of test cases whose result differs from the expected result

Modify ASCEND_HOME_DIR in the run.sh file.

ASCEND_HOME_DIR indicates the installation path of the CANN package. Change it based on the actual situation.

# Point to the installation address of the Ascend software package and export environment variables.
if [ ! $ASCEND_HOME_DIR ]; then
    export ASCEND_HOME_DIR=${INSTALL_DIR}     
fi
source $ASCEND_HOME_DIR/bin/set_env.bash

Go to the directory where the script file for executing the test framework is located and run the following command to verify the test framework code on the board:

bash run.sh <kernel_name> <soc_version> <core_type> <run_mode>

**Table 2** Script parameters
Parameter	Description	Value
<kernel_name>	Name of the Ascend C operator implementation file	For example, if the implementation file of the Add operator is add_custom.cpp, pass add_custom.
<soc_version>	Model of the AI processor where the operator runs.	For the Atlas training products and Atlas inference products, set ascendxxxyy based on the actual model. NOTE: Products except for Atlas A3 training products/Atlas A3 inference products: Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy. If Ascendxxxyy is the code sample path, you need to set ascendxxxyy. Atlas A3 training products/Atlas A3 inference products: Run the npu-smi info -t board -i id -c chip_id command on the server where the Ascend AI Processor is installed to obtain Chip Name and NPU Name. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. If Ascendxxx_1234 is the code sample path, you need to set ascendxxx_1234. Note that: id: device ID, which is the NPU ID obtained by running the npu-smi info -l command. *chip_id: chip ID, which is obtained by running the npu-smi info -m* command.
<core_type>	AI Core or Vector Core on which the operator runs	AiCore or VectorCore
<run_mode>	The operator runs in CPU or NPU mode.	cpu or npu

After the script is executed, information similar to the following is displayed. If succeed is displayed, the board verification is complete.

INFO: compile op on npu succeed!
[INFO] Succeeded to exec acl api aclrtCreateContext(&context, deviceId)
[INFO] Succeeded to exec acl api aclrtCreateStream(&stream)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&xHost), xByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&xDevice, xByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtMemcpy(xDevice, xByteSize, xHost, xByteSize, ACL_MEMCPY_HOST_TO_DEVICE)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&yHost), yByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&yDevice, yByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtMemcpy(yDevice, yByteSize, yHost, yByteSize, ACL_MEMCPY_HOST_TO_DEVICE)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&zHost), zByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&zDevice, zByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtSynchronizeStream(stream)
[INFO] Succeeded to exec acl api aclrtMemcpy(zHost, zByteSize, zDevice, zByteSize, ACL_MEMCPY_DEVICE_TO_HOST)
[INFO] aclrtDestroyStream successfully.
INFO: execute op on npu succeed!

Parent topic: msOpST (Operator Test)