QP Memory Resource Allocation (EI0011)

In the parameter plane connection setup phase, HCCL creates a QP. If the device memory is insufficient, an OOM error is reported. You can solve this problem by adjusting the service configuration, reducing the number of RoCE connections, or free some memory.

Symptom

The printed logs contain the keyword EI0011 or "Resource_Error_Insufficient_Device_Memory," as shown below:

[PID: 2103452] 2025-11-03-20:18:46.447.213 Resource_Error_Insufficient_Device_Memory(EI0011): Failed to allocate [size: [0.25MB, 3MB], Affected by QP depth configuration.] bytes of NPU memory.
        Possible Cause: Allocation failure due to insufficient NPU memory.
        Solution: Stop unnecessary processes and ensure the required memory is available.

Solution

Solve this problem by adjusting the service configuration (for example, batchSize), reducing the number of RoCE connections, or freeing some memory.

Notes:

If an OOM error is reported during allocation of other HCCL memory, such as cclBuffer memory, the drv component reports an error code and message. You can determine whether the cclBuffer memory allocation fails based on the error information or stack in the CANN logs. If the cclBuffer memory allocation fails, configure the HCCL_BUFFSIZE environment variable to adjust the size of the allocated memory.

Parent topic: Parameter Plane Connection Setup Phase