使用成都智算中心notebook进行模型训练,由于notebook多卡训练需要配置rank_table,运行hccl_tools.py生成json,出现如下报错:
[ma-user hccl_tools]$python hccl_tools.py --device_num "[0,8)"
start hccl_tools.py
visible_devices:['0', '1', '2', '3', '4', '5', '6', '7']
server_id:172.16.0.77
device_num_list: [0, 1, 2, 3, 4, 5, 6, 7]
/bin/sh: hccn_tool: command not found
Failed to call hccn_tool, try to read /etc/hccn.conf instead
Failed to read /etc/hccn.conf
Traceback (most recent call last):
File "hccl_tools.py", line 116, in main
device_ips[str(device_id)] = ret[0].split(":")[1].replace('\n', '')
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hccl_tools.py", line 120, in main
with open('/etc/hccn.conf', 'r') as fin:
PermissionError: [Errno 13] Permission denied: '/etc/hccn.conf'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hccl_tools.py", line 163, in <module>
main()
File "hccl_tools.py", line 128, in main
raise SystemError("Failed to find information for hccl")
SystemError: Failed to find information for hccl
使用成都智算中心notebook进行模型训练,由于notebook多卡训练需要配置rank_table,运行hccl_tools.py生成json,出现如下报错:
[ma-user hccl_tools]$python hccl_tools.py --device_num "[0,8)"
start hccl_tools.py
visible_devices:['0', '1', '2', '3', '4', '5', '6', '7']
server_id:172.16.0.77
device_num_list: [0, 1, 2, 3, 4, 5, 6, 7]
/bin/sh: hccn_tool: command not found
Failed to call hccn_tool, try to read /etc/hccn.conf instead
Failed to read /etc/hccn.conf
Traceback (most recent call last):
File "hccl_tools.py", line 116, in main
device_ips[str(device_id)] = ret[0].split(":")[1].replace('\n', '')
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hccl_tools.py", line 120, in main
with open('/etc/hccn.conf', 'r') as fin:
PermissionError: [Errno 13] Permission denied: '/etc/hccn.conf'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hccl_tools.py", line 163, in <module>
main()
File "hccl_tools.py", line 128, in main
raise SystemError("Failed to find information for hccl")
SystemError: Failed to find information for hccl