Application Scenario

The shape of operator input data come in two types, dynamic shape and static shape.

For static-shape operator development, a static shape size is specified at build time while the shape size does not need to be specified at run time. When developing a static-shape operator, its source code supports multiple static shapes. However, only one static shape can be specified at build time.

For dynamic-shape operator development, the shape size is not specified at build time, while the actual shape size is determined at run time. The binary file generated after operator building allows more than one shape.

• Static shape has its advantages:
  1. The shape size is specified at build time, therefore the sizes of all tensors are predetermined, improving the memory utilization.
  2. At build time, strategic optimization can be implemented based on the actual shape size.
  3. Ascend AI Processor shows better performance in parallel instruction execution than logic computation. In static-shape scenarios, scalar operations (including loop computation) are computed in at build time instead of the run time. The runtime performance may compromise due to too many scalar operations.
  4. For static-shape building, the instruction arguments are constants rather than variables. Therefore, the tiling size is fixed, and the compiler will not insert extra synchronization instructions. In this way, instructions can still be executed in parallel, preserving the execution performance.

• Static shape has its disadvantages:

  Each static shape requires one exclusive operator binary file, which increases the operator management complexity.