Why Am I Seeing the Message "IFMR node. Name:'layer_ifmr_op' Status Message: std::bad_alloc "?

Symptom

During PTQ calibration, when a layer is being quantized, a message reads "IFMR node Name:'layer_ifmr_op' Status Message: std::bad_alloc".

Possible Cause

The data volume involved in quantization is too large for the CPU or GPU storage in the environment, leading to a memory error.

Solution

1. Decrease the value of batch_size or batch_num to use a smaller calibration dataset. Alternatively, try again with the minimum result of batch_size times batch_num.
2. If only the feature map of a specific layer is memory-consuming, which causes the memory error, skip quantizing this layer.