Model Hyperparameters

Objective

The hyperparameter values in use are identical to those set in the benchmark model.

Principle

The hyperparameters set in the ported script may differ from those set in the benchmark model.

Ensure that the hyperparameters in use are the same as those set in the benchmark model.

Common issues include:

• During model porting of distributed training, the global batch size is incorrectly calculated based on the single-device batch size. As a result, the global batch size on the NPU is different from that of the benchmark model.
• Similar problems happen to the global learning rate.

Procedure

1. Review the hyperparameters in use.
2. Check the hyperparameter configuration files.
3. Compare the hyperparameters by debugging the benchmark model script and the ported script or printing the hyperparameter values.
4. Compare the learning rates in use.