Required Knowledge
This document provides guidance for you to develop high-performance operators on the Ascend AI Processor based on the Ascend C programming language. Before reading this document, you should:
- Be proficient in using the C++ programming language.
- Understand computer architecture.
- Know about hardware architecture of Ascend AI Processors.
- Have completed learning Ascend C documents and courses.
- Be able to set up the Ascend C development and debugging environments.
- Be able to independently develop Ascend C operators.
- Be proficient in using the profiling tool to obtain profile data.
For the preceding learning materials, click here.
Problems Solving
The operators mentioned in this document refer to those developed by using Ascend C. After operators are developed, you can further optimize operator performance with the help of this document.
This document begins with the hardware features and data interaction of heterogeneous computing, then introduces debugging and tuning ideas along with performance optimization methods for programming with Ascend C, and finally presents specific performance optimization cases.
Optimizing operator performance is a continuous iteration process, which goes through the following steps cyclically until the performance goal is achieved. These four steps are also the basis for you to understand the best practices.
This main content of this document consists of:
- Heterogeneous Computing: describes the operator deployment on hardware and the running data flow. It aims to help you understand the processes that may affect the operator execution performance in the hardware architecture.
- SIMD Operator Implementation: describes operator tiling and kernel implementation in three typical scenarios—vector programming, cube programming, and fusion operator programming—as specific applications of Ascend C programming paradigms.
- SIMT Operator Implementation: describes reference cases for operator development using SIMT APIs.
- SIMD and SIMT Hybrid Operator Implementation: describes the reference cases for operator development using the Reg Vector Computation and SIMT APIs for SIMD and SIMT programming.
- Function Debugging: describes some common scenarios that affect operator functions. It aims to enable you to quickly solve function problems and optimize performance.
- Performance Analysis: describes operator profiling. It aims to enable you to analyze profile data and identify the performance optimization direction. More details are introduced in Performance Tuning Tool.
- SIMD Operator Performance Optimization: describes performance optimization methods. It aims to enable you to optimize operator performance based on bottlenecks. The optimization suggestions include transfer optimization, memory optimization, API usage optimization, pipeline optimization, and tiling optimization. Some suggestions are a comprehensive reflection of the preceding classifications and are described in the sections that are closely related. These suggestions are classified by priority based on the performance effect and scope. Higher-priority suggestions are those that bring performance benefits to most Ascend C operators, while lower-priority suggestions are methods that affect only specific situations. You are not required to be familiar with all optimization methods, but can adopt proper optimization methods based on the operator performance bottlenecks obtained through analysis and gradually understand the whole optimization strategy.
- SIMD and SIMT Operator Performance Optimization: describes performance optimization suggestions for SIMD and SIMT hybrid programming scenarios, including memory access optimization and computing optimization.
- Best Practices: describes practical cases of operator performance optimization. It aims to help you better understand the preceding content, so that you can refer to the optimization methods and ideas to optimize the operator performance.