Experiment Parameters

ALLOW_HF32

This parameter is reserved and is not supported in the current version.

Sets whether to enable the function of automatically replacing the float32 data type with the HF32 data type. In the current version, this parameter takes effect only for Conv and Matmul operators. The corresponding options parameter is ge.exec.allow_hf32.

HF32 is a single-precision floating-point type developed by Ascend for internal computation of operators. The following shows the comparison with other common data types. HF32 shares the value range with float32, but its mantissa precision (11 bits) is close to FP16 (10 bits). Replacing the original float32 single-precision data type with the HF32 single-precision data type by precision reduction can greatly reduce the space occupied by data and improve performance.

Parameter values:

• true: Enables automatic conversion from FP32 to HF32 for Conv and Matmul operators.

  For details about the involved operators, see the opp/built-in/op_impl/ai_core/tbe/impl_mode/allow_hf32_matmul_t_conv_t.ini file under the CANN installation directory. This file cannot be modified by users.

• false: Disables automatic conversion from FP32 to HF32 for Conv and Matmul operators.

  For details about the involved operators, see the opp/built-in/op_impl/ai_core/tbe/impl_mode/allow_hf32_matmul_f_conv_f.ini file under the CANN installation directory. This file cannot be modified by users.

Default: For Conv operators, FP32-to-HF32 conversion is enabled by default. For Matmul operators, this conversion is disabled by default.

Restrictions:

• For the same operator, if enable_hi_float_32_execution or enable_float_32_execution is configured using OP_PRECISION_MODE, you are advised not to use this parameter together with ALLOW_HF32. If they are used together, the priority is as follows:

  OP_PRECISION_MODE(ByNodeName) > ALLOW_HF32 > OP_PRECISION_MODE(ByOpType)

• ALLOW_HF32 automatically replaces float32 with HF32. To make this parameter take effect, ensure that the input or output type of the involved operator is float32. The default value of PRECISION_MODE_V2 is fp16. If the operator type in the original network model is float32, the operator type is forcibly converted to float16. In this case, ALLOW_HF32 does not take effect. You are advised to change the value of PRECISION_MODE_V2 to origin. The default value of PRECISION_MODE is force_fp16, and you are advised to change the value to must_keep_origin_dtype or force_fp32.

Configuration example:

{ge::ir_option::ALLOW_HF32, "true"}

Applicable products:

• Ascend 950PR/Ascend 950DT: supported
• Atlas A3 training products/Atlas A3 inference products: supported
• Atlas A2 training products/Atlas A2 inference products: supported
• Atlas 200I/500 A2 inference products: not supported
• Atlas inference products: not supported
• Atlas training products: not supported
• IPV350: not supported

OO_CONSTANT_FOLDING

Extended parameter for debugging. It cannot be used in the production environment and will be released as a formal function in later versions.

Sets whether to enable constant folding optimization. The corresponding options parameter is ge.oo.constantFolding.

Constant folding is the process of replacing nodes that can be evaluated to a constant output value in a computational graph with that constant, and simplifying the structure of the computational graph accordingly.

Parameter values:

• true (default): enabled
• false: disabled

Configuration example:

{ge::ir_option::OO_CONSTANT_FOLDING, "true"}

Applicable products:

All processors

OO_DEAD_CODE_ELIMINATION

Extended parameter for debugging. It cannot be used in the production environment and will be released as a formal function in later versions.

Sets whether to enable dead-edge elimination optimization. The corresponding options parameter is ge.oo.deadCodeElimination.

Dead-edge elimination (switch dead-edge elimination): When pred (input 1) of a switch statement is a constant node, one of the branches can be eliminated based on the value of const. If const is true, the false branch is eliminated; if const is false, the true branch is eliminated.

Parameter values:

• true (default): enabled
• false: disabled

Configuration example:

{ge::ir_option::OO_DEAD_CODE_ELIMINATION, "true"}

Applicable products:

All processors

OO_LEVEL

Extended parameter for debugging. It cannot be used in the production environment and will be released as a formal function in later versions.

Sets multi-level optimization options for graph build, including subgraph optimization, entire graph optimization, and static shape model offloading. The corresponding options parameter is ge.oo.level.

Static shape model offloading: In this approach, the input and output shapes of all operators in a static shape model can be determined at build time, allowing for model-level memory orchestration and operator tiling computation to be completed on the host. These computations are then batched and sent to the device stream when the model is loaded, but they are not executed immediately. Instead, the execution of all tasks within the model is triggered by the delivery of model execution tasks.

Parameter values:

• O1: Disables all graph fusion and UB fusion passes, and performs only optimizations related to static offloading, such as InferShape (output tensor shape inference), constant folding, dead-edge elimination, and other optimizations.

  The Ascend 950PR/Ascend 950DT does not support UB fusion. Only the graph fusion pass is disabled.

• O3 (default): Enables all optimizations.

Restrictions:

If the value is O1, all graph fusion and UB fusion passes are disabled, and only passes related to static offloading are enabled. However, the graph fusion passes in the following files are enabled by default because function problems may occur if they are disabled:

All graph fusion passes under the ExceptionalPassOfO1Level field in the ${INSTALL_DIR}/<arch>-linux/lib64/plugin/opskernel/fusion_pass/config/fusion_config.json file

Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann. <arch> indicates the OS architecture.

Configuration example:

{ge::ir_option::OO_LEVEL, "O3"}

Applicable products:

All processors

TUNE_DEVICE_IDS

Not supported in the current version.