--modify_mixlist

Applicability

Description

In the mixed precision scenario, you can modify the mixed precision blocklist, trustlist, and graylist of operators to specify operators that allow precision reduction and operators that do not allow.

For the blocklist, trustlist, and graylist, you can view the value of the flag parameter under precision_reduce in the built-in optimization policy file of ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/config/xxx/aic-xxx-ops-info-*.json. (Select Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.xxx based on the actual product.)

• true (trustlist): Precision reduction is allowed in mixed precision mode.
• false (blocklist): Precision reduction is not allowed in mixed precision mode.
• Not specified (graylist): Operators on the graylist follow the same precision processing as its upstream operator.

See Also

Method for enabling mixed precision:

• Set --precision_mode to allow_mix_precision, allow_mix_precision_bf16, or allow_mix_precision_fp16.
• Set --precision_mode_v2 to mixed_float16 or mixed_bfloat16 or mixed_hif8.

  Do not use this option together with --precision_mode. You are advised to use --precision_mode_v2.

Argument

Argument: Directory (including the file name) of the mixed precision list file.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.

Restrictions:

• The list is in *.json format. The operator list in the file is specified by users. Multiple operators must be separated by commas (,).
• The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by AI processors?.

Suggestions and Benefits

None

Example

The following is an example of querying the blocklist, trustlist, and graylist. true indicates the trustlist, false indicates the blocklist, and if flag is not set, it indicates the graylist.

"Conv2D":{
    ......
    "precision_reduce":{
        "flag":"true"
     },
    ......
}

The following is an example of the mixed precision list. ops_info.json is an example file name, and OpTypeA, OpTypeB, OpTypeC, and OpTypeD are example operators.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

{
"black-list": {                  // Blocklist
     "to-remove": [                // Move an operator from the blocklist to the graylist. Ensure that the specified operator is already on the blocklist.
     "OpTypeA"
     ],
     "to-add": [                   // Move an operator from the trustlist or graylist to the blocklist.
     "OpTypeB"
     ]
  },
  "white-list": {                  // Trustlist
     "to-remove": [                // Move an operator from the trustlist to the graylist. Ensure that the specified operator is already on the trustlist.
     "OpTypeC"
     ],
     "to-add": [                   // Move an operator from the blocklist or graylist to the trustlist.
     "OpTypeD"
     ]
  }
}

• To move an operator from the trustlist to the blocklist, take either of the following method:
  1. Add the operator to the blocklist.

     1 2 3 4 5

     { "black-list": { "to-add": ["A"] } }

     The operator will be removed from the trustlist to the blocklist.

  2. Remove the operator from the trustlist and add it to the blocklist.

     1 2 3 4 5 6 7 8

     { "black-list": { "to-add": ["A"] } "white-list": { "to-remove": ["A"] } }

     The operator will be moved from the trustlist to the blocklist.

• If an operator is simply removed from the blocklist or trustlist, it will be added to the graylist. See the following example:

  1 2 3 4 5

  { "white-list": { "to-remove": ["A"] } }

  The operator will be removed from the trustlist to the graylist.

Upload the configured ops_info.json file to any directory (for example, $HOME/module) on the server where ATC is located.

--precision_mode=allow_mix_precision  --modify_mixlist=$HOME/module/ops_info.json

Dependencies and Restrictions

None