Single-Operator Board Test Framework Generation

Specify the ST case definition file (.json) and implementation file kernel_name.cpp of the Ascend C operator to automatically generate the board test framework for calling the kernel function, test and verify the operator, and view the output result to check whether the operator function is correct.

This function supports only and Atlas Training Series Product, but not .
The addr and tiling attributes cannot be specified for any parameter.
The #ifndef __CCE_KT_TEST__ can be used to encapsulate kernel function calls.

Prepare the following input files:
- ST case definition file (*.json file) of the operator.
- Operator implementation file on the kernel (*.cpp file). For details, see Operator Implementation on Kernel.
Generate the test code to call the kernel function. For details about the parameters, see Table 3.
```
msopst ascendc_test -i xx/OpType_case.json -kernel xx/add_custom.cpp -out ./output_data
```

View the execution result.

After the command is executed, the message "Process finished!" is displayed. A timestamp directory is generated in the directory specified by -out. In the timestamp directory, a folder named after OpType of the operator is generated for storing test cases and test results. The directory structure is as follows:

 {time_stamp}
│   ├── OpType
│   │   ├── CMakeLists.txt     // Compilation script
│   │   ├── data    
│   │   │   └── xx.bin        
│   │   │   └── xx.bin
│   │   ├── data_utils.h
│   │   ├── main.cpp        // Test framework
│   │   └── run.sh      // Script file for invoking the test framework
│   └── st_report.json        // Execution report

After the command is executed successfully, an st_report.json report is automatically generated. The report records the detailed test information and the runtime states in each phase, which facilitates troubleshooting and test result comparison.

The st_report.json file is saved in The st_report saved in.

2024-01-17 08:40:55 (3271037) - [INFO] Create 1 sub test cases for Test_AddCustom_001.
2024-01-17 08:40:55 (3271037) - [INFO] [STEP2] [data_generator.py] Generate data for testcase.
2024-01-17 08:40:55 (3271037) - [INFO] Start to generate the input data for Test_AddCustom_001_case_001_ND_float.
2024-01-17 08:40:55 (3271037) - [INFO] Generate data for testcase in $HOME/AddCustom/output/20240117084055/AddCustom/data.
2024-01-17 08:40:55 (3271037) - [INFO] [STEP3] [gen_ascendc_test.py] Generate test code of calling of kernel function for AscendC operator.
2024-01-17 08:40:55 (3271037) - [INFO] Content appended to $HOME/AddCustom/output/20240117084055/AddCustom/main.cpp successfully.
2024-01-17 08:40:55 (3271037) - [INFO] AscendC operator test code files for kernel implement have been successfully generated.
2024-01-17 08:40:55 (3271037) - [INFO] If you want to execute kernel function in Ascend aihost or cpu, please execute commands: cd $HOME/AddCustom/output/20240117084055/AddCustom && bash run.sh [KERNEL_NAME](add_custom) [SOC_VERSION](ascendxxxyy) [CORE_TYPE](AiCore/VectorCore) [RUN_MODE](cpu/npu). For example: cd $HOME/AddCustom/output/20240117084055/AddCustom && bash run.sh add_custom ascendxxxyy AiCore npu
2024-01-17 08:40:55 (3271037) - [INFO] Process finished!
2024-01-17 08:40:55 (3271037) - [INFO] The st report saved in: $HOME/AddCustom/output/20240117084055/st_report.json.

**Table 1** st_report.json description
Field			Description
run_cmd	-	-	Command
report_list	-	-	List of reports of test cases.
	trace_detail	-	Execution details.
		st_case_info	Test information, including the following: expect_data_path: path of the expected result case_name: test case name input_data_path: path of input data planned_output_data_paths: path of the actual result op_params: operator parameter information
		stage_result	Result information in each runtime phase, including the following: status: running status of the phase, either success or failure result: output result stage_name: stage name cmd: command
	case_name	-	Test name
	status	-	Actual test result, either success or failure
	expect	-	Expected test result, either success or failure
summary	-	-	Summary of comparison of the actual test result and the expected test result.
	test case count	-	Number of test cases
	success count	-	Number of test cases whose result is the same as the expected result
	failed count	-	Number of test cases whose result differs from the expected result

Modify ASCEND_HOME_DIR in the run.sh file.

ASCEND_HOME_DIR indicates the installation path of the CANN package. Change it based on the actual situation.

# Point to the installation address of the Ascend software package and export environment variables.
if [ ! $ASCEND_HOME_DIR ]; then
    export ASCEND_HOME_DIR=${INSTALL_DIR}     
fi
source $ASCEND_HOME_DIR/bin/setenv.bash

Go to the directory where the script file for executing the test framework is located and run the following command to verify the test framework code on the board:

bash run.sh <kernel_name> <soc_version> <core_type> <run_mode>

**Table 2** Script parameters
Parameter	Description	Value
<kernel_name>	Name of the Ascend C operator implementation file	For example, if the implementation file of the Add operator is add_custom.cpp, pass add_custom.
*<soc_version>*	Model of the AI processor where the operator runs.	For the Atlas Training Series Product and , set ascendxxxyy based on the actual model. NOTE: Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy. If Ascendxxxyy is the code sample path, you need to set ascendxxxyy.
<core_type>	AI Core or Vector Core on which the operator runs	AiCore or VectorCore
<run_mode>	The operator runs in CPU or NPU mode.	cpu or npu

After the script is executed, information similar to the following is displayed. If "succeed" is displayed, the board verification is complete.

INFO: compile op on npu succeed!
[INFO] Succeeded to exec acl api aclrtCreateContext(&context, deviceId)
[INFO] Succeeded to exec acl api aclrtCreateStream(&stream)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&xHost), xByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&xDevice, xByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtMemcpy(xDevice, xByteSize, xHost, xByteSize, ACL_MEMCPY_HOST_TO_DEVICE)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&yHost), yByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&yDevice, yByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtMemcpy(yDevice, yByteSize, yHost, yByteSize, ACL_MEMCPY_HOST_TO_DEVICE)
[INFO] Succeeded to exec acl api aclrtMallocHost((void**)(&zHost), zByteSize)
[INFO] Succeeded to exec acl api aclrtMalloc((void**)&zDevice, zByteSize, ACL_MEM_MALLOC_HUGE_FIRST)
[INFO] Succeeded to exec acl api aclrtSynchronizeStream(stream)
[INFO] Succeeded to exec acl api aclrtMemcpy(zHost, zByteSize, zDevice, zByteSize, ACL_MEMCPY_DEVICE_TO_HOST)
[INFO] aclrtDestroyStream successfully.
INFO: execute op on npu succeed!

Parent topic: msOpST (Operator Test)