AI CPU Operator UT

Prerequisites

The custom operator has been developed, including the operator implementation code and operator prototype definition. For details, see Operator Code Implementation (AI CPU) and Operator Prototype Definition.
The lcov dependency has been installed. For GCC 8 and earlier versions, install the lcov dependency of version 1.14. For GCC 9 and later versions, install the lcov dependency of version 1.16.
CMake 3.14 or later has been installed. If the CMake version does not meet the requirement, upgrade the CMake.

The CentOS 7.8 Arm container does not support the UT.

Procedure

Create UT cases.

A UT case file can be created in either of the following ways:
Right-click the root directory of the operator project and choose New Cases > AI CPU UT Case from the shortcut menu.

If UT cases of the operator exist, right-click the testcases or testcases > ut directory, and choose New Cases > AI CPU UT Case from the shortcut menu to create UT cases.

In the Create UT for an Operator window, choose the target operator, and click OK, as shown in the following figure.

If the UT cases of the operator already exist, the message "testcases/ut/ops_test/xx already exists. Do you want to overwrite?" is displayed.

You can choose Overwrite to overwrite the current test cases or Cancel to cancel the overwriting.

After the creation is complete, the testcases folder is generated in the root directory of the operator project. The directory structure is as follows:

├── MyOperator           // Root directory of the project
│   ├──  testcases                            
│   │   ├── libs                 // GTest framework. It is a third-party dependency and can be ignored.
│   │   ├──  ut                              
│   │   │   ├── aicpu_test
│   │   │   │   ├── reshape_cust   
│   │   │   │   │   ├── CMakeLists.txt       // Build script
│   │   │   │   │   ├── test_reshape_cust_impl.cc      // Test case file for the operator implementation code
│   │   │   │   │   ├── test_reshape_cust_proto.cc    // Test case file for the operator prototype definition code. Operators of the MindSpore framework do not contain this file.
│   │   │   │   ├── CMakeLists.txt            // Build script
│   │   │   │   ├── test_main.cc             // Main entry for test case calling
│   │   │   ├ CMakeLists.txt

During NeoKylin and Kylin OS running, add the following information in bold to the ./testcases/aicpu_test/reshape_cust/CMakeLists.txt file. For other OSs, skip this step.

cmake_minimum_required(VERSION 3.14)
set(CMAKE_CXX_FLAGS "-std=c++11")

...

link_directories(
            "${ATC_DIR}/lib64"
            "${GTEST_DIR}"
           "/usr/local/gcc7.3.0/lib64/"  // Set this parameter to the actual lib64 path of gcc7.3.0 if the user configures the Dockerfile.
    )

    set(CUSTOM_OBJECT_NAME "reshape_cust_proto_test")
    add_executable(${CUSTOM_OBJECT_NAME}
            ${PROJECT_DIR}/testcases/ut/aicpu_test/test_main.cc
            ${OP_PROTO_SOURCE_SRCS}
            ${OP_PROTO_TEST_FILES})

    target_link_libraries(${CUSTOM_OBJECT_NAME} gtest c_sec alog pthread error_manager graph register)
endif()

The GCC version must be 7.3.0 or later for the AI CPU UT. If the GCC version does not meet the requirement, upgrade the GCC.

Write UT cases in C++ for the operator implementation code.
In the testcases/ut/aicpu_test/reshape_cust/test_reshape_cust_impl.cc file, write UT cases in C++ for the operator implementation code. Execute the operator and compare the compute result with the expect result to test the operator logic.
Write UT cases in C++ for the operator prototype definition. This is not supported by the MindSpore framework.
In the testcases/ut/aicpu_test/reshape_cust/test_reshape_cust_proto.cc file, write the UT case in C++ for the operator prototype definition. Define the operator instance, update the input and output of the operator, call InferShapeAndType, and verify the execution process as well as the result of InferShapeAndType.

Run the UT cases of the operator implementation file.

You can run the UT cases of all operators in the current project or run the UT cases of a single-operator.

Right-click the testcases/ut/aicpu_test folder and choose Run AI CPU Operator 'All' UT Impl with coverage from the shortcut menu to run the test cases of the operator implementation code in the folder.
Right-click the testcases/ut/aicpu_test/operator_name folder and choose Run AI CPU Operator 'operator_name' UT Impl with coverage to run test cases of implementation code of a single operator.
Right-click the testcases/ut/ops_test/operator_name/test_operator_name_impl.cc file and choose Run AI CPU Operator 'operator_name' UT Impl with coverage to run test cases of implementation code of a single operator.

When the cases run for the first time, the corresponding configuration dialog box is displayed. Configure the parameters and click Run. For details about how to modify the run configurations, see Editing a Run Task Configuration.

Figure 1 Run configurations

**Table 1** Run configurations
Parameter	Description
Name	Name of the run configuration (user-defined).
Test Type	ut_impl is recommended.
Compute Unit	Compute unit. AI Core/Vector Core AI CPU You can select different compute units to switch between the AI Core/Vector Core and AI CPU UT configuration pages. Currently, only the AI CPU is supported.
Operator Name	Test cases to run. all: all test cases Other values: test cases of a specific operator
Case Names	Test cases to run, that is, the C++ UT cases of the operator implementation code. You can select all or some test cases.

Check the execution result.

After the execution is complete, view the execution result in the Run log print window at the bottom. See Figure 2.
Figure 2 Execution result
In the Run window, click the URL in report index.html to view the UT case coverage. In the URL, localhost indicates the server IP of MindStudio.

To view the test results of UT cases, you need to use a browser. If no browser is available, install one.

If "Page'http://***.html'requested without authorization, you can copy URL and open it in browser to trust it." is displayed, rectify the fault by referring to Analysis Sample of Timeline-based Tuning of AI CPU Operators.
Click the operator on the HTML page to navigate to the UT case coverage dialog box, as shown in Figure 3. The green and red labels are used to indicate the coverage rates.
Figure 3 UT coverage details

(Optional) Debug UT of the operator implementation file.

Right-click the testcases/ut/aicpu_test/operator name folder and choose Debug AI CPU Operator operator nameUT Impl to debug test cases of implementation code of a single-operator.

For details about how to perform debugging, see Project Debugging and Execution. For details about how to set shortcut keys for debugging, see Others.

When the case runs for the first time, the corresponding configuration dialog box is displayed. Configure the parameters and click Debug. For details about how to modify the run configurations, see Editing a Run Task Configuration.

**Table 2** Run configuration
Parameter	Description
Name	Name of the run configuration (user-defined).
Test Type	ut_impl is recommended.
Compute Unit	Compute unit. Select the AI CPU.
Operator Name	Test cases to run. all: all test cases Other values: test cases of a specific operator
Case Names	Test cases to run, that is, the C++ UT cases of the operator implementation code. You can select all or some test cases.

Check the debugging information.
You can perform debugging, view the debugging information, and view the debugging result in the Debug window in the lower part of the page.

Run the UT cases defined in the operator prototype.

You can run the UT cases of all operators in the current project or run the UT cases of a single-operator.

Right-click the testcases/ut/aicpu_test folder and choose Run AI CPU Operator 'All' UT Proto from the shortcut menu to run the test cases of the operator prototype definition code in the folder.
Right-click the testcases/ut/aicpu_test/operator_name folder and choose Run AI CPU Operator 'operator_name' UT Proto from the shortcut menu to run test cases of prototype definition code of a single operator.
Right-click the testcases/ut/ops_test/operator_name/test_operator_name_proto.cc file and choose Run AI CPU Operator 'operator_name' UT Proto from the shortcut menu to run test cases of prototype definition code of a single operator.

When the case runs for the first time, the corresponding configuration dialog box is displayed. Configure the parameters and click Run. For details about how to modify the run configurations, see Editing a Run Task Configuration.

Figure 4 Configuration page

**Table 3** Run configuration
Parameter	Description
Name	Name of the run configuration (user-defined).
Test Type	ut_proto is recommended.
Compute Unit	Compute unit. AI Core/Vector Core AI CPU You can select different compute units to switch between the AI Core/Vector Core and AI CPU UT configuration pages. Currently, only the AI CPU is supported.
Operator Name	Test cases to run. all: all test cases Other values: test cases of a specific operator
Case Names	Select test cases to be executed, that is, the test cases defined in TEST_F. You can select all or some test cases.

After the execution is complete, view the execution result in the log print window at the bottom, which contains the test case execution success and failure information. See Figure 5.

Figure 5 Execution result

After the Ascend-CANN-Toolkit is updated, an old UT project may fail to be executed. To solve this problem, delete the old UT dependency folder and execute the UT again. The old UT dependency folder is under AI CPU operator project directory/testcases/libs/aicpu_depend

Others

The AI CPU operator UT supports single-step debugging. To perform single-step debugging, click in the upper left corner of the project page. On the top menu bar that is displayed, choose File > Settings > Keymap > Main menu > Run > Debugging Actions to set the debugging shortcut keys.

Parent topic: UT