Documentation

Designing Fusion Patterns

The fusion pattern design should specify the fusion solution, fusion operator prototype, and fusion mapping relationship.

Designing the Fusion Solution

The following uses the many-to-one fusion pattern DecodeBboxV2ScopeFusionPass as an example to describe how to design and develop a scope fusion pattern.

This fusion pattern aims to fuse all small operators under Decode scope into a large operator DecodeBboxV2. The scope contains two Exp operators, four Mul operators, four Sub operators, a multiple of two RealDiv operators, two Unpack operators, one Pack operator, three Transpose operators, and does not contain any Softmax operator, as shown in Figure 1.

Figure 1 Fusion diagram

Specifying the Prototype of the Fused Operator

The prototype of the DecodeBboxV2 fused operator is as follows.

REG_OP(DecodeBboxV2)
    .INPUT(boxes, TensorType({DT_FLOAT16,DT_FLOAT}))
    .INPUT(anchors, TensorType({DT_FLOAT16,DT_FLOAT}))
    .OUTPUT(y, TensorType({DT_FLOAT16,DT_FLOAT}))
    .ATTR(scales, ListFloat, {1.0, 1.0, 1.0, 1.0})
    .ATTR(decode_clip, Float, 0.0)
    .ATTR(reversed_box, Bool, false)
    .OP_END_FACTORY_REG(DecodeBboxV2)

For details about the operator prototype APIs, see "GE Namespace > Prototype Definition (REG_OP)" in Basic Data Structures and APIs.

Specifying the Fusion Relationship

As shown in Figure 1:

  • Input 0 of transpose is used as input 0 boxes of DecodeBboxV2.
  • Input 0 of get_center_coordinates_and_sizes/transpose is used as input 1 anchors of DecodeBboxV2.
  • The output of transpose_1 is used as the output y of DecodeBboxV2.