What Is Scope Fusion?

To implement high-performance computing, small operators in a subgraph are usually fused to a larger operator (many-to-one fusion) or a composite of small operators (many-to-many fusion) to make full use of hardware acceleration resources.

In the TensorFlow framework, you can use the scope function tf.name_scope() of TensorFlow to place different objects and operations in scopes specified by tf.name_scope() so that the logical relationship can be clearly displayed on TensorBoard.

import tensorflow as tf;   
tf.reset_default_graph()

# Define an area named xx_name_scope and work in it.
with tf.name_scope('xx_name_scope'):   
     a = tf.constant(1,name='my_a')    
     b = tf.Variable(2,name='my_b') 
     c = tf.add(a,b,name='my_add')      
print("a.name = "+a.name) 
print("b.name = "+b.name) 
print("c.name = "+c.name)

# Save the graph for TensorBoard drawing.
with tf.Session() as sess:
   sess.run(tf.global_variables_initializer())
   writer = tf.summary.FileWriter("./test",sess.graph)
   print(sess.run(c))
writer.close()

Returns:

a.name = xx_name_scope/my_a:0 
b.name = xx_name_scope/my_b:0 
c.name = xx_name_scope/my_add:0

As shown in the returns, xx_name_scope is added before the name attribute of all objects and operations under tf.name_scope(), indicating that the content is within the scope.

You can also view the scope information on TensorBoard.

Scope fusion is a scope-based fusion capability. Small operators in the scope are replaced with one larger operator or a composite of operators to improve efficiency.

In addition to the fusion of all operators within the scope, the Ascend platform also supports more flexible fusion configuration, such as nested identification, parallel identification, and scope+common operator identification, as shown in Figure 1.