Overview

The main goal in optimizing the traditional model is to improve both accuracy and performance together, as shown in Table 1.

Table 1 Tuning description

Optimization Scenario

Optimization Objective

Description

Accuracy optimization

To improve the accuracy and stability of the model.

In model debugging, most accuracy problems are caused by operator accuracy problems. The main causes of operator accuracy anomalies include precision overflow, operator implementation differences, improper fusion patterns, and hardware differences. You need to analyze and solve these issues based on specific symptoms.

Performance optimization

To improve the computing efficiency and achieve optimal resource allocation.

In single-node inference scenarios, performance bottlenecks mainly occur in scheduling and computing. You can use msProf and MindStudio Insight to quickly locate the problem type, analyze data in depth, and adjust the model implementation or optimize the system scheduling policy to improve inference performance.