Scope Fusion Implementation Solution

The scope fusion implementation solution involves registration, creation, and execution of scope fusion patterns.

Solution Overview

ScopePass Register: registers fusion patterns.
OpParser-Plugin Register: registers parsers of fusion operators and maps the TensorFlow operators in a scope to fusion operators adapted to AI processor. The mapping of common operators is also implemented by this module.

Registering a Scope Fusion Pattern

During ATC model conversion or graph execution in the TensorFlow framework, the system loads the fusion rule plugin SO files (including all built-in fusion rules and custom fusion rules) into the ScopePass Register. In the subsequent parser execution phase, you can determine whether to create and execute a fusion pattern based on the enabling status of the fusion pattern.

Figure 1 Scope fusion pattern registration workflow

Creating and Executing a Scope Fusion Pattern

Figure 2 Scope fusion pattern execution workflow

During model parsing, TensorFlow Parser invokes the capability provided by ScopePassManager to represent the node and scope information in GraphDef using the data type of scope fusion and generate a ScopeGraph. The layers from top to bottom are ScopeGraph, ScopeTree, and Scope.
The TensorFlow Parser creates and executes scope fusion, and saves the final matching result to the ScopeGraph, including the following actions in sequence:
1. Creates scope fusion patterns based on the enabling status of the registered fusion patterns.
2. Obtains the created scope fusion patterns, and matches the patterns one by one according to the registration sequence of the fusion patterns. If a pattern is matched, the corresponding scope type is set.
  
  If the name of a custom fusion pattern is the same as that of a built-in fusion pattern, the custom fusion pattern is used for matching.
3. Filters the scopes matched in the previous step based on the connections of the scopes. For example, the scopes obtained in the previous step may not be the final fusion scopes. You need to filter out the scopes with parallel relationships or the scopes with nesting relationships.
4. Sets the fusion result for the finally matched scopes, including the name, type, input, output, and description of the fusion nodes.
5. Saves the fusion result to the ScopeGraph generated in step 1.
TensorFlow Parser performs operations such as adding nodes, connecting nodes, and constructing the IR graph based on the fusion result.

Key Data Structures

Data structures such as ScopeGraph and ScopeTree provide diverse capabilities and data storage functions required for Scope fusion. You can view the API definition in Scope Fusion Pattern Development APIs. For details about the functionality of the key classes, see Table 1.

**Table 1** Functionality of key classes
Class	Function	See Also
Scope	Defines the scope type and attribute.	Class Scope
ScopeTree	A tree structure that stores all scope information. You can query the sub-scope and node information related to the scope.	Class ScopeTree
ScopeGraph	Includes ScopeTree, defines the member functions for scope matching calculation, and also defines the final result of scope fusion identification.	Class ScopeGraph
ScopePattern	Currently, scope patterns are classified into the following types: NodeOpTypeFeature: Scopes are matched based on the number of operators or the multiple of a number of operators of a certain type in the scope. NodeAttrFeature: Scopes are matched based on the value of an attribute of an operator of a certain type in the scope. ScopeFeature: Scopes are matched based on the features of the scope or its sub-scopes.	Class ScopePattern Class NodeOpTypeFeature Class NodeAttrFeature Class ScopeFeature
ScopeBaseFeature	Base class of the preceding three patterns, which defines the basic operations of the three patterns.	Class ScopeBaseFeature
ScopeBasePass	Base class of custom fusion patterns, which provides the API definition, common execution process, and implementation of the common pattern matching process.	Class ScopeBasePass
ScopesResult	Saves the scopes left after further matching and filtering.	Class ScopesResult
FusionScopesResult	Saves the fusion result, including the fused operator's name, type, inputs, outputs, description, and internal operator combination information (for many-to-many scenarios).	Class FusionScopesResult
ScopeUtil	Provides common tool functions.	Class ScopeUtil
ScopeAttrValue	Data structure used in the NodeAttrFeature pattern definition, which is used to define attribute-related patterns.	Class ScopeAttrValue

Parent topic: Basic Concepts