Documentation

Fields in the AI CPU ST Operator Test Case Definition File

Table 1 AI CPU operator ST case definition file (.json)

Field

Description

Test_xxx_xx

-

Mandatory. A string.

Test case name.

Input[xx]

-

Mandatory.

Operator input.

NOTICE:

The number of parameter values in each input of an operator must be the same. Otherwise, test case generation fails.

For example, if the Input[01] supports two formats, the Input[02] also supports two formats.

Similarly, the parameter values of type, shape, data_distribute, and value_range in all Input[xx] must be the same.

-

Name

Mandatory.

Operator input name, which is of the string type.

If the expectation function is configured, the operator input name must be the same as that input in the expectation function.

-

Format

Mandatory.

A string or a 1D array of strings.

Format of the input tensor. Must not be empty.

Common data formats are as follows:

  • NCHW
  • NHWC
  • ND: any format.
  • NC1HWC0: a 5D format defined by Huawei. C0 is closely related to the micro-architecture, and the value is equal to the Cube Unit size, for example, 16. C1 is obtained by dividing the C dimension by C0, that is, C1 = C/C0. When the division is not exact, the last data segment is padded to C0.
  • FRACTAL_Z: a format of the convolution weight.
  • FRACTAL_NZ: fractal format defined by Huawei. The data format of the output matrix is NW1H1H0W0 during Cube Unit computation. The matrix is divided into (H1 x W1) fractals in column-major order, which looks like an N-shape layout. Each fractal consists of (H0 x W0) elements in row-major order, resembling a z-shaped layout. Therefore the NW1H1H0W0 format is referred to as the Nz format. (H0 x W0) indicates the size of a fractal, as shown in the following figure.

  • RESERVED: reserved. If this value is used, type needs to be set to UNDEFINED, indicating that the corresponding operator input is optional.

Note that the numbers of formats, types, and shapes must be the same.

-

OriginFormat

Optional. Original tensor format.

If this parameter is not carried, the original tensor format is retained.

-

Type

Mandatory.

A string or a 1D array of strings.

Input data type.

  • bool
  • int8
  • uint8
  • int16
  • uint16
  • int32
  • int64
  • uint32
  • uint64
  • float16
  • float
  • double
  • complex64
  • complex128
  • UNDEFINED: applies to an optional input.

Note that the numbers of formats, types, and shapes must be the same.

-

Shape

Mandatory.

An int or a 1D or 2D array of ints.

Input tensor shape.

Example:

[8, 3, 256, 256] or [[1, 3, 256, 256], [8, 3, 256, 256]].

If you enter an invalid shape, for example, [0], an error is reported.

Note that the numbers of formats, types, and shapes must be the same.

-

OriginShape

Optional. Original tensor shape.

If this field is not carried, the original tensor shape is retained.

-

Value

Optional.

Path string or tensor array.

If you need to specify the input data, you can add the Value field. There are two configuration methods:

Enter tensor data, for example, [1,2,3,4].

"Value": [1,2,3,4]

Specify the path of a binary file, for example, test.bin.

"Value": "home/HiHwAiUser/projects/test.bin"

Prepare the .bin file by yourself. You can enter an absolute path. If the Value field is added, the DataDistribute and ValueRange fields will be hidden. As such, ensure that the value of the Value field is consistent with those of Format, Type, and Shape fields.

-

ValueRange

Mandatory.

An int or float, or a 1D or 2D array of ints or floats.

Value range. Must not be left empty.

Formatted as [min_value, max_value], with min_value <= max_value.

-

DataDistribute

Mandatory.

A string or a 1D array of strings.

Data distribution mode for generating test data.
  • uniform: returns random values that are evenly distributed.
  • normal: returns random values of the normal distribution (Gaussian distribution).
  • beta: returns random values of Beta distribution.
  • laplace: returns random values of Laplace distribution.
  • triangular: returns random values of the triangular distribution.
  • relu: returns random values that are evenly distributed and activated by the ReLU function.
  • sigmoid: returns random values that are evenly distributed and activated by the sigmoid function.
  • softmax: returns random values that are evenly distributed and activated by softmax function.
  • tanh: returns random values that are evenly distributed and activated by tanh function.

-

isConst

Optional.

The value is of the Bool type. The default value is false.

  • true: configures test cases with constant inputs.
  • false: configures test cases with tensor inputs.

Output[xx]

-

Mandatory.

Operator output.

NOTICE:

The number of values in Output must be the same as that in Input. Otherwise, test case generation fails.

For example, if Input supports two formats, the Output also supports two formats.

-

Name

Mandatory.

Operator input name, which is of the string type.

If the expectation function is configured, the operator output name must be the same as that output from the expectation function.

-

Format

Mandatory.

A string or a 1D array of strings.

Format of the output tensor. Must not be empty.

Common data formats are as follows:

  • NCHW
  • NHWC
  • ND: any format.
  • NC1HWC0: a 5D format defined by Huawei. C0 is closely related to the micro-architecture, and the value is equal to the Cube Unit size, for example, 16. C1 is obtained by dividing the C dimension by C0, that is, C1 = C/C0. When the division is not exact, the last data segment is padded to C0.
  • FRACTAL_Z: a format of the convolution weight.
  • FRACTAL_NZ: fractal format defined by Huawei. The data format of the output matrix is NW1H1H0W0 during Cube Unit computation. The matrix is divided into (H1 x W1) fractals in column-major order, which looks like an N-shape layout. Each fractal consists of (H0 x W0) elements in row-major order, resembling a z-shaped layout. Therefore the NW1H1H0W0 format is referred to as the Nz format. (H0 x W0) indicates the size of a fractal, as shown in the following figure.

Note that the numbers of formats, types, and shapes must be the same.

-

OriginFormat

Optional. Original tensor format.

If this parameter is not carried, the original tensor format is retained.

-

Type

Mandatory.

A string or a 1D array of strings.

Output data type.

  • bool
  • int8
  • uint8
  • int16
  • uint16
  • int32
  • int64
  • uint32
  • uint64
  • float16
  • float
  • double
  • complex64
  • complex128

Note that the numbers of formats, types, and shapes must be the same.

-

Shape

Mandatory.

An int or a 1D or 2D array of ints.

Input tensor shape.

Example:

[8, 3, 256, 256] or [[1, 3, 256, 256], [8, 3, 256, 256]].

If you enter an invalid shape, for example, [0], an error is reported.

Note that the numbers of formats, types, and shapes must be the same.

-

OriginShape

Optional. Original tensor shape.

If this field is not carried, the original tensor shape is retained.

Attribute

-

Optional.

-

Name

Required if attr is configured.

A string.

Attribute name. Must not be empty.

-

Type

Required if attr is configured. Attribute data type.

A string.

Output data type.

  • bool
  • int
  • float
  • string
  • list_bool
  • list_int
  • list_float
  • list_string
  • list_list_int

-

Value

Required if attr is configured. Must not be null.

A string.

Attribute value. Varies according to type.

  • bool: true/false
  • int: 10
  • float: 1.0
  • string: "NCHW"
  • list_bool: [false, true]
  • list_int: [1, 224, 224, 3]
  • list_float: [1.0, 0.0]
  • list_string: ["str1", "str2"]
  • list_list_int: [[1, 3, 5, 7], [2, 4, 6, 8]]

Expected Result Verification

-

Optional.

-

Script Path

Path of the test function for generating expected operator result.

-

Script Function

Name of the test function for generating expected operator result.

K-Level Test Cases

-

Optional.

-

Fuzz Script Path

Optional.

Path of the string type.

Relative or absolute path of the fuzz testing script.

-

Fuzz Case Num

Optional.

An int.

Number of test cases generated using a script. The value range is [1, 2000].

-

Fuzz Function

Optional.

A string.

Name of the function for generating fuzzing input in the fuzzing script. The default value is fuzz_branch.
  • When inputx.paramType in the operator information library definition file (.ini) is set to optional, set format of inputx to UNDEFINED or RESERVED, and type to UNDEFINED in the generated operator test cases.
  • In the operator information library definition file (.ini), if a tensor implementation uses a Huawei-developed format and is different from the original format, you need to manually enter the original format and shape in OriginFormat and OriginShape to convert the format and shape into those compatible with the offline model.
    • OriginFormat is an array specifying the formats supported by the original operator. The array must have the same length as the Format array.
    • The value of OriginShape must correspond to the value of Shape and match the values of Format and OriginFormat.
  • All parameters except Type of name and Attr in Input[xx], Output[xx], and Attr support fuzz input.
Parent topic: Reference