Precision Comparison

BUFFER_OPTIMIZE

Sets whether to enable buffer optimization. The corresponding options parameter is ge.bufferOptimize.

Parameter values:

• l1_optimize: Enables L1 optimization. This option is invalid in the current version and is equivalent to off_optimize.
• l2_optimize (default): Enables L2 optimization.
• off_optimize: Disables buffer optimization.

l1 indicates L1 buffer, which is general internal storage. L1 buffer offers a large data transfer area in the AI Core. It can temporarily store data that needs to be repeatedly used in the AI Core, thereby reducing the frequency of data reads from or writes to the bus. l2 indicates L2 buffer, which is external storage. An AI Core loads data from the external storage to the internal storage for computation.

Suggestions:

You are advised to enable buffer optimization. This function can improve computing efficiency and performance. However, it is possible that your model contains an operator that is not yet covered by the current implementation, which affects the precision. You can disable data buffer optimization when the precision is affected. If the precision meets the requirement after the data cache optimization function is disabled, identify the faulty operator and contact technical support for further analysis and resolution. After the operator problem is resolved, you are advised to enable the data cache optimization function.

Configuration example:

{ge::ir_option::BUFFER_OPTIMIZE, "l2_optimize"}

Applicable products:

All processors

FUSION_SWITCH_FILE

The corresponding options parameter is ge.fusionSwitchFile.

Sets the directory (including the file name) of the configuration file for the fusion pattern. The directory can contain letters, digits, underscores (_), hyphens (-), and periods (.).

The built-in graph fusion and UB fusion patterns are enabled by default. You can disable specified fusion patterns in the configuration file. Some fusion patterns cannot be disabled due to functionality restrictions. For the full list of fusion patterns that can be disabled, see Graph Fusion and UB Fusion Patterns.

The Ascend 950PR/Ascend 950DT does not support UB fusion.

Configuration example:

The following is a template of the fusion_switch.cfg configuration file. on indicates that a fusion pattern is enabled, and off indicates that a fusion pattern is disabled.

{
    "Switch":{
        "GraphFusion":{
            "RequantFusionPass":"on",
            "ConvToFullyConnectionFusionPass":"off",
            "SoftmaxFusionPass":"on",
            "NotRequantFusionPass":"on",
            "SplitConvConcatFusionPass":"on",
            "ConvConcatFusionPass":"on",
            "MatMulBiasAddFusionPass":"on",
            "PoolingFusionPass":"on",
            "ZConcatv2dFusionPass":"on",
            "ZConcatExt2FusionPass":"on",
            "TfMergeSubFusionPass":"on"
        },
        "UBFusion":{
            "TbePool2dQuantFusionPass":"on"
        }
    }
}

If UB fusion is not supported, UBFusion can be deleted from the configuration file.

To disable all fusion patterns at a time, refer to this configuration file example.

{
    "Switch":{
        "GraphFusion":{
            "ALL":"off"
        },
        "UBFusion":{
            "ALL":"off"
         }
    }
}

If UB fusion is not supported, UBFusion can be deleted from the configuration file.

Note:

1. Disabling certain fusion patterns may cause functional issues. Therefore, this one-click disable operation only disables some system-level fusion patterns instead of all fusion patterns.
2. To disable all fusion patterns except selected ones, refer to the following example.
   Configuration file example:

   {
       "Switch":{
           "GraphFusion":{
               "ALL":"off",
               "SoftmaxFusionPass":"on"
           },
           "UBFusion":{
               "ALL":"off",
               "TbePool2dQuantFusionPass":"on"
           }
       }
   }

   If UB fusion is not supported, UBFusion can be deleted from the configuration file.

Applicable products:

All processors