Checklist

Before you start accuracy tuning, use the following checklist to exclude errors with the benchmark model or model porting process.

• Benchmark model script check

  Item	Description	Result
  Constant Validation Accuracy	The benchmark model must produce consistent results across multiple training runs. If the benchmark model does not meet this requirement, the model is not suitable as an accuracy benchmark.	Passed/Failed/Not checked
  Mixed Precision Training	As the AI processor (or NPU) hardware architecture supports only mixed precision training for the user model, the user model needs to be trained with mixed precision. If mixed precision training is not enabled or not enabled successfully for the user model, the NPU may fail to train the model or the accuracy of the trained model may not meet the expectation.	Passed/Failed/Not checked

• Ported script check

  Table 1 Pre-tuning checklist

  Item	Description	Result
  Mixed Precision Training on the NPU	The model is successfully ported to the NPU before accuracy tuning. Distributed training (if involved) is enabled. Especially, mixed precision training is enabled during model porting.	Passed/Failed/Not checked
  Loss Scaling on NPU	Loss scaling must be enabled in the script ported to the NPU. Generally, the LossScaleManager parameters need to be configured, as the NPU differs from the GPU in mixed precision computing.	Passed/Failed/Not checked
  Dataset Processing	Ensure dataset integrity. Training datasets are typically large and prone to incompleteness.	Passed/Failed/Not checked
  Data Preprocessing	The data preprocessing pipeline in your model code may contain variables that are automatically set based on resources. Such variables can lead to inconsistent randomness in dataset shuffling. Check the API calls related to data preprocessing in the code to minimize the difference.	Passed/Failed/Not checked
  Shard Method	The data preprocessing part of the user model code may shard datasets to different nodes based on file name or number of files. This results in significant sharding discrepancies, or even duplicate file shards being assigned to different nodes, because the file read API sorts file names differently across nodes. Add debugging code to exclude such problems, ensuring the sharding policy consistent with that of the benchmark model.	Passed/Failed/Not checked
  Training Procedure	During training, process errors such as failing to clear intermediate data often occur, which may produce unwanted intermediate data, leading to inconsistent accuracy. Get familiar with the training process and check your training and validation steps.	Passed/Failed/Not checked
  Model Hyperparameters	The hyperparameters set in the ported script may differ from those set in the benchmark model. Ensure that the hyperparameters in use are the same as those set in the benchmark model.	Passed/Failed/Not checked