Cast: Direct-Cast Quantization Algorithm

Cast is the simplest quantization algorithm. It requires no calibration data and directly converts high-precision floating-point numbers (Float16/BFloat16) to the HiFloat8 format. The conversion process is as follows:

1. Activations: directly converted from Float16 or BFloat16 to HiFloat8.
2. Weights: The maximum weight value is collected offline and scaled to fit the representation range of HiFloat8.

This algorithm applies to scenarios that demand high deployment speed, feature relatively concentrated model data distribution, and can tolerate considerable accuracy loss. Its advantages are as follows:

• No calibration data or extra computation is required, enabling the simplest deployment.
• It delivers the fastest inference speed with no additional overhead.
• Benefiting from the large dynamic range and high-precision characteristics of HiFloat8, direct conversion from Float16 or BFloat16 incurs minor accuracy loss.