Documentation

Introduction

Overview

Scope fusion is a scope-based fusion capability that replaces small operators in a scope with one larger operator or a composite of operators to improve efficiency.

This document describes the built-in scope fusion patterns. A collection of APIs is opened to developers to customize scope fusion patterns. You can find detailed description about the APIs in TensorFlow Parser Scope Fusion Pattern Developer Guide.

The supported TensorFlow version is 1.15.

General and Non-general Fusion Patterns

The fusion patterns are classified into general and non-general fusion patterns.

  • General scope fusion patterns are applicable to all networks. They are enabled by default and cannot be manually disabled.
  • Non-general scope fusion patterns are applicable to specific networks. They are disabled by default and can be manually enabled.
    Table 1 Enabling a non-general fusion pattern

    Scenario

    Method

    TensorFlow model conversion using ATC for offline inference

    Include the enable_scope_fusion_passes option to set the fusion pattern (or fusion patterns separated by commas) to take effect.

    --enable_scope_fusion_passes = DecodeBboxV2ScopeFusionPass

    TensorFlow model parsing for offline inference

    Parse a TensorFlow model by using the aclgrphParseTensorFlow call.

    Set ENABLE_SCOPE_FUSION_PASSES to the fusion pattern (or fusion patterns separated by commas) to take effect.

    {ge::AscendString(ge::ir_option::ENABLE_SCOPE_FUSION_PASSES), ge::AscendString("DecodeBboxV2ScopeFusionPass")},

    Training or online inference within the TensorFlow framework

    Set enable_scope_fusion_passes to the fusion pattern (or fusion patterns separated by commas) to take effect within the TensorFlow framework.

    import tensorflow as tf
from npu_bridge.estimator import npu_ops
from tensorflow.core.protobuf.rewriter_config_pb2 import RewriterConfig

config = tf.ConfigProto()
custom_op = config.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name = "NpuOptimizer"
custom_op.parameter_map["use_off_line"].b = True
custom_op.parameter_map["enable_scope_fusion_passes"].s = tf.compat.as_bytes("DecodeBboxV2ScopeFusionPass")
config.graph_options.rewrite_options.remapping = RewriterConfig.OFF

with tf.Session(config=config) as sess:
  sess.run(xx_name_scope) # xx_name_scope is an example of the fused operator name.