What Do I Do If "IFMR node. Name:'layer_ifmr_op' Status Message: std::bad_alloc" Is Displayed During the Calibration?

Symptom

When a layer is quantized during PTQ calibration, a memory error "IFMR node. Name:'layer_ifmr_op' Status Message: std::bad_alloc" is reported.

Possible Cause

The data size involved in quantization is too large for the CPU or GPU storage in the environment, leading to a memory error.

Solution

1. Decrease the value of batch_size or batch_num to use a smaller calibration dataset. Alternatively, try again with the minimum result of batch_size times batch_num.
2. If only the feature map of a specific layer is memory-consuming, which causes the memory error, skip quantizing this layer.