Overview

Prerequisites

Install the software package. Specifically, prepare a hardware environment with the AI processor and install the driver, firmware, and CANN package. For details, see CANN Software Installation.
GCC version: 9.5.0 (recommended) or later.
CMake version: 3.20.0 (recommended) or later.
Install MSPTI. You can obtain MSPTI in either of the following ways (this dependency needs to be installed when the PGO feature is enabled):
- You can directly obtain a compiled release package from https://gitcode.com/Ascend/mspti/releases. The installation command is as follows:
```
./mindstudio-profiler-tools-interface_{version}_{arch}.run --install --install-path={cann_install_path}
```
  --install-path: specifies the installation path, which must be the installation path of the preceding CANN software package.
- Source code compilation:
  - Run the following command to clone the source code repository to your local host (using the HTTPS protocol as an example):
```
git clone https://gitcode.com/Ascend/mspti.git
```
  - Run the following command to perform compilation:
```
bash script/build.sh
```
  - Replace the CANN software package file:
    After the compilation is successful, copy the csrc/include header file directory to {cann_install_path}/cann/tools/mspti/include and the build/libmspti.so dynamic library directory to {cann_install_path}/cann/tools/mspti/lib64. {cann_install_path} must be the installation path of the CANN software package.
After the installation is complete, set environment variables.
After the CANN software is installed, log in to the environment as the CANN operating user and run the source ${INSTALL_DIR}/set_env.sh command to set environment variables. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

Enabling AutoFuse

In the TensorFlow graph mode, you can enable automatic fusion by setting environment variables. These variables control functions and DFX. For automatic fusion out-of-the-box, only function control needs to be enabled. DFX control is used for fault locating or optimization. The environment variable for function control is AUTOFUSE_FLAGS, and that for DFX control is AUTOFUSE_DFX_FLAGS. The environment variable value is configured in the format of "--key=value". Each key represents a control point. Use semicolons (;) to separate multiple control points.

Currently, the following types of operators support automatic fusion. The methods for enabling them are described in the table below.

**Table 1** Operator types supported by automatic fusion
Fusion Type	How to Enable	Auto Tiling Algorithm	Auto Tiling Algorithm Precision	Intermediate Files Generated During Fusion Operator Generation	PGO Algorithm
Elemwise	Enabled by default	The axis sorting algorithm (default)	High-precision solution (default)	Not retained by default	The core control algorithm (default)
Broadcast	Enabled by default	The axis sorting algorithm (default)	High-precision solution (default)	Not retained by default	The core control algorithm (default)
Reduce	Explicit control using environment variables	The axis sorting algorithm (default)	High-precision solution (default)	Not retained by default	The core control algorithm (default)
Concat	Explicit control using environment variables	The axis sorting algorithm (default)	High-precision solution (default)	Not retained by default	The core control algorithm (default)

The following uses the Reduce and Concat operators as examples to describe how to enable the automatic fusion function:

Function control:

export AUTOFUSE_FLAGS="--enable_autofuse=true;--autofuse_enable_pass=reduce,concat"

DFX control:

export AUTOFUSE_DFX_FLAGS="--att_accuracy_level=1;--att_profiling=true"

For details about each of the preceding environment variables, see AUTOFUSE_FLAGS Environment Variable and Its Control Points and AUTOFUSE_DFX_FLAGS Environment Variable and Its Control Points.

Automatic fusion can be enabled in the offline inference static shape scenario using the following steps:

Set an environment variable to enable automatic fusion. For example:
```
export AUTOFUSE_FLAGS="--enable_autofuse=true"
```
- Use the ATC (offline model compiler) to convert the model into an offline model (OM). (In the automatic fusion scenario, single-operator-to-OM conversion is not supported, that is, the --singleop parameter is not supported.)
- Use the GE API aclgrphBuildModel to compile the model and generate an offline model (OM). (In the automatic fusion scenario, single-operator-to-OM conversion is not supported, that is, the INSERT_OP_FILE parameter is not supported.)
Model loading and inference
Use the acl API to load the generated OM model and perform inference.

For details about how to use the ATC tool, see Ascend Tensor Compiler (ATC).

For details about how to use the GE API, see Building a Graph into an Offline Model.

For details about the acl API, see Application Development (C & C++) > Model Management.

Parent topic: AutoFuse Enabling Method