Replacing LossScaleOptimizer

Precautions

• For the Atlas Training Series Product , skip this step if your script does not involve the use of LossScaleOptimizer. Otherwise, port the script by referring to this section.

Description

Generally, LossScaleOptimizer is used to prevent numeric underflow in mixed precision mode. In the saturation mode, as a floating-point range error on the NPU is reported as a global error instead of returning Inf or NaN, you are advised to use npu.train.optimizer.NpuLossScaleOptimizer provided by the NPU to obtain the correct overflow/underflow detection result.

The usage of npu.train.optimizer.NpuLossScaleOptimizer is the same as that of tf.keras.mixed_precision.LossScaleOptimizer.

Replace the occurrences of tf.keras.mixed_precision.LossScaleOptimizer in your script with npu.train.optimizer.NpuLossScaleOptimizer directly. If your script uses a different form of LossScaleOptimizer, import it to tf.keras.mixed_precision.LossScaleOptimizer and validate the functionality and quality before replacement.