Using ST Samples

Overview

You can tweak the operator test sample code provided in the Ascend samples repository on the command line to adapt to your own operator test code.

Go to the Ascend samples repository on Gitee or GitHub and download the sample package that matches the required version. For the version mapping, see "Release Notes" in the README file. Find the sample in the cplusplus/level1_single_api/4_op_dev/2_verify_op directory.

Implementing ST Cases

The procedure of implementing ST cases includes: define the operator test cases (.json), convert a custom operator into an offline model file (.om) of a single-operator, load the offline model using AscendCL APIs, pass operator input data, execute the operator, and then view the verification result. The procedure detail is shown in the following figure.

Figure 1 Custom operator ST workflow

The following describes how to implement ST cases for a custom operator based on the sample code.

Configuring the JSON Test Case File

The .json file of a custom operator describes the operator in the aspects of its inputs, outputs, and attributes. Refer to Operator Prototype Definition to configure the operator .json file. The following is an example.

[
  {
    "op": "Conv2D",
    "input_desc": [
      {
        "format": "NCHW",
        "shape": [8, 512, 7, 7],
        "type": "float16"
      },
      {
        "format": "NCHW",
        "shape": [512, 512, 3, 3],
        "type": "float16"
      }
    ],
    "output_desc": [
      {
        "format": "NCHW",
        "shape": [8, 512, 7, 7],
        "type": "float16"
      }
    ],
    "attr": [
      {
        "name": "strides",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      },
      {
        "name": "pads",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      },
      {
        "name": "dilations",
        "type": "list_int",
        "value": [1, 1, 1, 1]
      }
    ]
  }
]

For details about the parameters in the custom operator JSON file and more configuration examples, see "Advanced Features > Generating a Single-Operator Model" in ATC Instructions.

Generating a Single-Operator Model File Using ATC

Set the basic environment variables required by the CANN software by referring to Environment Setup.

During model conversion using ATC, the TBE Python library required for operator building in python/site-packages under the CANN software installation directory will be automatically added to the PYTHONPATH environment variable.

If more Python dependencies in addition to the TBE Python library are imported in the operator implementation file, add the environment variable PYTHONPATH and configure the paths of these imported Python dependencies in the following format:

export PYTHONPATH=xxxx:$PYTHONPATH
Use the ATC tool to load the single-operator description file (.json file) to generate a single-operator offline model.
atc --singleop=test_data/config/xxx.json --soc_version=${soc_version} --output=op_models
- singleop: specifies the operator description file (.json), which is the path relative to where the atc command is executed.
- soc_version: specifies the version of Ascend AI Processor in use.
- output: specifies the path of the generated model file, which is the path relative to where the atc command is executed.
For details about other parameters of the ATC tool, see "Conversion Example" in ATC Instructions.

Constructing the Input Data

Before implementing the AscendCL code for single-operator verification, create a binary file of the operator test data and name it input_x.bin, where x is an integer starting from 0, for example, input_0.bin and input_1.bin.

Note: You can generate test data by using the random function of NumPy and export the data to a binary.

Implementing Operator Test Code in AscendCL

Construct an operator description object.

The template requires manual tweaks.

Modify the CreateOpDesc() function in the src/main.cpp file to construct an operator description object.

The following is an example of constructing opDesc of the Add operator without attributes:

OperatorDesc CreateOpDesc()
{
    std::string opType = "Add";          // Operator type
    // Construct the input and output description of the operator.
    std::vector<int64_t> shape{8, 16};
    aclDataType dataType = ACL_INT32;
    aclFormat format = ACL_FORMAT_ND;
    OperatorDesc opDesc(opType);
    opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format);
    opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format);
    opDesc.AddOutputTensorDesc(dataType, shape.size(), shape.data(), format);
    return opDesc;
}

The following is an example of constructing opDesc of the ConcatD operator with attributes:

OperatorDesc CreateOpDesc()
{
    std::string opType = "ConcatD";
    std::vector<int64_t> shape1{1, 1, 4, 4};
    std::vector<int64_t> shape2{1, 2, 4, 4};
    aclDataType dataType = ACL_INT32;
    aclFormat format = ACL_FORMAT_ND;
    OperatorDesc opDesc(opType);
    opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);
    opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);
    opDesc.AddOutputTensorDesc(dataType, shape2.size(), shape2.data(), format);
    // Construct operator attributes.
    auto opAttr = opDesc.opAttr;
    aclopSetAttrInt(opAttr, "N", 2);
    aclopSetAttrInt(opAttr, "concat_dim", 1);
    return opDesc;
}

The following is an example of constructing opDesc of the Conv2D operator:

OperatorDesc CreateOpDesc()
{
    std::vector<int64_t> shape{8, 512, 7, 7};
    std::string opType = "Conv2D";
    aclDataType dataType = ACL_FLOAT16;
    aclFormat format = ACL_FORMAT_NCHW;
    OperatorDesc opDesc(opType);
    opDesc.AddInputTensorDesc(dataType, shape.size(), shape.data(), format); // The first input of the Conv2D operator
    std::vector<int64_t> shape1{512,512,3,3};  
    opDesc.AddInputTensorDesc(dataType, shape1.size(), shape1.data(), format);   // The second input of the Conv2D operator
    std::vector<int64_t> shape2{8, 512, 7, 7};
    opDesc.AddOutputTensorDesc(dataType, shape2.size(), shape2.data(), format);  // Output of the Conv2D operator
    int64_t intList[4]{1, 1, 1, 1};
    auto opAttr = opDesc.opAttr;
    aclopSetAttrListInt(opAttr, "strides", 4, intList);
    aclopSetAttrListInt(opAttr, "pads", 4, intList);
    aclopSetAttrListInt(opAttr, "dilations", 4, intList);
    return opDesc;
}

OperatorDesc is the operator description object constructed in acl_execute_add/inc/operator_desc.h.

std::string opType: operator type
std::vector<const aclTensorDesc *> inputDesc: input description of the operator
std::vector<const aclTensorDesc *> outputDesc: output description of the operator
aclopAttr *opAttr: operator attribute
Use the aclopSetAttr**() API to construct attributes. The aclopSetAttr* API to be called varies according to the attribute type. For details, see the description of the aclopAttr class in "AscendCL API Reference > Data Types and Operations" of Application Software Development Guide (C&C++)Application Software Development Guide (C&C++).
OperatorDesc &AddInputTensorDesc(aclDataType dataType, int numDims, const int64_t *dims, aclFormat format)
Constructs the operator input. If the operator has multiple inputs, state each input using a separate opDesc.AddInputTensorDesc() statement.
OperatorDesc &AddOutputTensorDesc(aclDataType dataType, int numDims, const int64_t *dims, aclFormat format)
Constructs the operator output. If the operator has multiple outputs, state each output using a separate opDesc.AddOutputTensorDesc() statement.

For details about the value ranges of aclDataType and aclFormat, see "AscendCL API Reference > Data Types and Operations" in Application Software Development Guide (C&C++).

Load and run the single-operator model.
The following key APIs are involved:

Call aclInit to initialize AscendCL and call aclrtSetDevice to specify the compute device.

Call aclopSetModelDir to set the directory of the single-operator model file (.om) to be loaded.

Call the AscendCL memory management APIs to allocate memory for storing the operator inputs and outputs.

Allocate memory for storing the input and output data of the operator based on the constructed operator description information (including the input and output tensor description and operator attributes), and call aclopExecuteV2 to load and execute the operator.

Call aclrtSynchronizeStream to wait for the stream tasks to complete.

Call aclrtDestroyStream to destroy the stream.

Call aclrtResetDevice to reset the device.

Call aclFinalize to deinitialize AscendCL.

The acl_execute_add/src/main.cpp file provides the common process sample code for loading and executing a single-operator model. You can use the sample code to load your single-operator model file and operator description object to verify the single-operator function.

Building and Running Test Cases

Test case building and execution include: build the test case code to generate a dynamic library file and a single-operator model file, as well as run test data in the operating environment. For details, see Samples.

Parent topic: Operator ST