--modify_mixlist

Applicable Products

Description

Edits the blocklist, trustlist, and graylist for mixed precision.

• The precision of operators on the trustlist will be reduced from float32 to float16 in mixed precision mode.
• The precision of operators on the blocklist will not be reduced from float32 to float16 in mixed precision mode. Their original precision float32 will be preserved.
• Operators on the graylist follow the same precision processing as its upstream operator.

See Also

Use this option together with --precision_mode and set the argument to allow_mix_precision, allow_mix_precision_bf16, or allow_mix_precision_fp16 to indicate that the mixed precision mode is used. In the mixed precision scenario, the precision of certain float32 operators in a network model is automatically reduced to float16 based on the built-in tiling policy. After --modify_mixlist is used, you can adjust the built-in tiling policy to specify the operators whose precision can be reduced and the operators whose precision cannot be reduced.

allow_mix_precision_bf16 supports the following models:

Atlas A2 training products/Atlas A2 inference products

Atlas A3 training products/Atlas A3 inference products

Arguments

Argument: directory (including the file name) of the mixed precision list file.

Format: The directory (including the file name) can contain letters, digits, underscores (_), hyphens (-), periods (.), and Chinese characters.

Restrictions:

• The list is in *.json format. The operator list in the file is specified by users. Multiple operators must be separated by commas (,).
• The operator type must be OpType of the Ascend IR–defined operator. For details about how to view the operator type, see How Do I Determine the Mapping Between Operators in the Original Network Model and Operators Supported by AI processor?.

Suggestions and Benefits

None

Examples

The following provides a template of the mixed precision list file (ops_info.json).

{
"black-list": {                  // Blocklist
     "to-remove": [                // Move an operator from the blocklist to the graylist. Ensure that the specified operator is already on the blocklist.
     "Yolo"
     ],
     "to-add": [                   // Move an operator from the trustlist or graylist to the blocklist.
     "Matmul"
     ]
  },
  "white-list": {                  // Trustlist
     "to-remove": [                // Move an operator from the trustlist to the graylist. Ensure that the specified operator is already on the trustlist.
     "Conv2D"
     ],
     "to-add": [                   // Move an operator from the blocklist or graylist to the trustlist.
     "Bias"
     ]
  }
}

• To move an operator from the trustlist to the blocklist, take either of the following method:
  1. Add the operator to the blocklist.

     {
       "black-list": { 
          "to-add": [A]
       }
     }

     The operator will be removed from the trustlist to the blocklist.

  2. Remove the operator from the trustlist and add it to the blocklist.

     {
       "black-list": {
          "to-add": [A]
       }
       "white-list": {
          "to-remove": [A]
       }
     }

     The operator will be moved from the trustlist to the blocklist.

• If an operator is simply removed from the blocklist or trustlist, it will be added to the graylist. See the following example:

  {
    "white-list": {
       "to-remove": [A]
    }
  }

  The operator will be removed from the trustlist to the graylist.

Upload the configured ops_info.json file to any directory (for example, ${HOME}/module) on the server where AOE is located.

--precision_mode=allow_mix_precision  --modify_mixlist=${HOME}/module/ops_info.json

Dependencies and Restrictions

None