Index Operator Out of Range

Symptom

The host application log file log/[run|debug]/plog/plog-pid_*.log contains the 0x800000 error about the index operator.

Common index operators include GatherV2 (the API corresponding to PyTorch is index_select), Scatter (the API corresponding to PyTorch is scatter_update), and GatherElements (the API corresponding to PyTorch is gather). Generally, the input names of this type of operators are index and indices.

Fault root causes

Due to performance reasons, operators of this type cannot perform index-bound checking. Therefore, the error cannot be correctly reported. When the index is out of range, the AI Core error of the 0x800000 may occur.

Solution

Take the GatherV2 operator as an example. The API of PyTorch corresponding to this operator is index_select:

torch.index_select(input, dim, index, *, out=None)

This operator involves parameters such as input, dim, and index. If 0x800000 is displayed for this operator, you can use the Python printing function to print the shape, dim, and index values of the input. Further, it is determined that the index is out of range. Search for the source of the index by viewing the code. The rules are as follows:

0 <= index[i] < input.shape[dim]