Scope Fusion Implementation Solution

Solution Overview

• ScopePass Register: registers fusion patterns.
• OpParser-Plugin Register: registers parsers of fusion operators and maps the TensorFlow operators in a scope to fusion operators adapted to Ascend AI Processor. The mapping of common operators is also implemented by this module.

Registering a Scope Fusion Pattern

During ATC model conversion or graph execution within the TensorFlow framework, the system automatically loads the fusion pattern plugin .so files (including all built-in fusion patterns and custom fusion patterns) in Ascend-CANN-Toolkit path/ascend-toolkit/latest/opp/built-in/framework/ to the ScopePass Register. In the subsequent parser execution phase, you can determine whether to create and execute a fusion pattern based on the enabling status of the fusion pattern.

Figure 1 Scope fusion pattern registration workflow

Creating and Executing a Scope Fusion Pattern

Figure 2 Scope fusion pattern execution workflow

1. During model parsing, the TensorFlow Parser invokes the capability provided by ScopePassManager to represent the node and scope information in GraphDef using the data type of scope fusion and generates a ScopeGraph. The layers from top to bottom are ScopeGraph, ScopeTree, and Scope.
2. The TensorFlow Parser creates and executes scope fusion, and saves the final matching result to the ScopeGraph, including the following actions in sequence:
   1. Creates scope fusion patterns based on the enabling status of the registered fusion patterns.
   2. Obtains the created scope fusion patterns, and matches the patterns one by one according to the registration sequence of the fusion patterns. If a pattern is matched, the corresponding scope type is set.
      

      If the name of a custom fusion pattern is the same as that of a built-in fusion pattern, the custom fusion pattern is used for matching.

   3. Filters the scopes matched in the previous step based on the connections of the scopes. For example, the scopes obtained in the previous step may not be the final fusion scopes. You need to filter out the scopes with parallel relationships or the scopes with nesting relationships.
   4. Sets the fusion result for the finally matched scopes, including the name, type, input, output, and description of the fusion nodes.
   5. Saves the fusion result to the ScopeGraph generated in step 1.
3. TensorFlow Parser performs operations such as adding nodes, connecting nodes, and constructing the IR graph based on the fusion result.

Key Data Structures