--modify_mixlist

Description

In the mixed precision scenario, you can modify the mixed precision blocklist, trustlist, and graylist of operators to specify operators that allow precision reduction and operators that do not allow.

For the blocklist, trustlist, and graylist, you can view the value of flag in the precision_reduce option in the built-in tuning policy file ${INSTALL_DIR}/opp/built-in/op_impl/ai_core/tbe/config/<soc_version>/aic-<soc_version>-ops-info.json.

• true (trustlist): Precision reduction is allowed in mixed precision mode.
• false (blocklist): Precision reduction is not allowed in mixed precision mode.
• Not specified (graylist): Operators on the graylist follow the same precision processing as its upstream operator.

See Also

Method for enabling mixed precision:

• Set --precision_mode to allow_mix_precision or allow_mix_precision_fp16.
• Set --precision_mode_v2 to mixed_float16.

  Do not use this option together with --precision_mode. You are advised to use --precision_mode_v2.

Argument

Argument: Directory (including the file name) of the mixed precision list file.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.

Restrictions:

• The list is in *.json format. The operator list in the file is specified by users. Multiple operators must be separated by commas (,).
• The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by Ascend AI Processors?.

Suggestions and Benefits

None

Example

The following is an example of querying the blocklist, trustlist, and graylist. true indicates the trustlist, false indicates the blocklist, and if flag is not set, it indicates the graylist.

"Conv2D":{
    "precision_reduce":{
        "flag":"true"
}

The following is an example of the mixed precision list. ops_info.json is an example file name, and OpTypeA, OpTypeB, OpTypeC, and OpTypeD are example operators.

{
"black-list": {                  // Blocklist
     "to-remove": [                // Move an operator from the blocklist to the graylist. Ensure that the specified operator is already on the blocklist.
     "OpTypeA"
     ],
     "to-add": [                   // Move an operator from the trustlist or graylist to the blocklist.
     "OpTypeB"
     ]
  },
  "white-list": {                  // Trustlist
     "to-remove": [                // Move an operator from the trustlist to the graylist. Ensure that the specified operator is already on the trustlist.
     "OpTypeC"
     ],
     "to-add": [                   // Move an operator from the blocklist or graylist to the trustlist.
     "OpTypeD"
     ]
  }
}

• To move an operator from the trustlist to the blocklist, take either of the following method:
  1. Add the operator to the blocklist.

     {
       "black-list": { 
          "to-add": [A]
       }
     }

     The operator will be removed from the trustlist to the blocklist.

  2. Remove the operator from the trustlist and add it to the blocklist.

     {
       "black-list": {
          "to-add": [A]
       }
       "white-list": {
          "to-remove": [A]
       }
     }

     The operator will be moved from the trustlist to the blocklist.

• If an operator is simply removed from the blocklist or trustlist, it will be added to the graylist. See the following example:

  {
    "white-list": {
       "to-remove": [A]
    }
  }

  The operator will be removed from the trustlist to the graylist.

Upload the configured ops_info.json file to any directory (for example, $HOME/module) on the server where ATC is located.

--precision_mode=allow_mix_precision  --modify_mixlist=$HOME/module/ops_info.json

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Dependencies and Restrictions

None