标红的报错
[ModelArts Service Log]2023-12-27 17:11:28,718 - ERROR - proc-rank-0-device-0 (pid: 346) has exited with non-zero code: 1
小白,,
想用华为的云平台运行以下昇腾的yolov4,使用的是平台提供的Ascend计算,单卡计算,就报这个错误。求好心人帮帮孩子的课程设计。
具体报错日志上下文
===finish data synchronization===
===save flag===
Finish sync data from s3://lwjwyq/pretrain/ to /cache/checkpoint_path.
Preload downloaded: ['cspdarknet53_backbone.ckpt']
from path: /home/ma-user/modelarts/outputs/train_url_0/
to path: /cache/train
INFO:root:No files to copy.
===finish data synchronization===
===save flag===
Finish sync data from /home/ma-user/modelarts/outputs/train_url_0/ to /cache/train.
Workspace downloaded: []
Zip file path: /cache/data/coco.zip
Unzip file save dir: /cache/data
This is not zip.
===Finish extract data synchronization===
Device: 0, Finish sync unzip data from /cache/data/coco.zip to /cache/data.
Traceback (most recent call last):
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 303, in <module>
run_train()
File "/home/ma-user/modelarts/user-job-dir/code/model_utils/moxing_adapter.py", line 105, in wrapped_func
run_func(*args, **kwargs)
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 181, in run_train
profiler = set_default()
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 59, in set_default
device_id=device_id, ascend_config={"precision_mode": "allow_fp32_to_fp16"})
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/_checkparam.py", line 1090, in wrapper
return func(*args, **kwargs)
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/_checkparam.py", line 1059, in wrapper
return func(*args, **kwargs)
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/context.py", line 921, in set_context
raise ValueError(f"For 'context.set_context', the keyword argument {key} is not recognized! For detailed "
ValueError: For 'context.set_context', the keyword argument ascend_config is not recognized! For detailed usage of 'set_context', please refer to the Mindspore official website.
[ModelArts Service Log]2023-12-27 17:11:28,718 - ERROR - proc-rank-0-device-0 (pid: 346) has exited with non-zero code: 1
[ModelArts Service Log]2023-12-27 17:11:28,718 - INFO - Begin destroy training processes
[ModelArts Service Log]2023-12-27 17:11:28,718 - INFO - proc-rank-0-device-0 (pid: 346) has exited before receiving the term signal
[ModelArts Service Log]2023-12-27 17:11:28,719 - INFO - End destroy training processes
time="2023-12-27T17:11:28+08:00" level=info msg="the run_train script is exiting..." Component=ShellScripts Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="the run_train 180 script is exited with 1" Component=ShellScripts Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="command is exit with 1" file="process.go:178" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="NPU training process exits with exit code 1, and the environment will be retained for 0s." file="process.go:226"
time="2023-12-27T17:11:28+08:00" level=info msg="the environment has been retained for 0s." file="process.go:228"
time="2023-12-27T17:11:28+08:00" level=info msg="DetectResult msg no error\n" file="process.go:206" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="start parse ascend log, current time is :2023-12-27 17:11:28" file="ascend_log_handler.go:129" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="ascend_fd binary found, path: /home/ma-user/modelarts/package/ascend-fd" file="executor.go:116" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="zombie process cleaner is exiting" file="cleaner_unix.go:53" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
The parse job starts. Please wait.
All parse job succeeded.
The parse job is complete.
标红的报错
[ModelArts Service Log]2023-12-27 17:11:28,718 - ERROR - proc-rank-0-device-0 (pid: 346) has exited with non-zero code: 1
小白,,
想用华为的云平台运行以下昇腾的yolov4,使用的是平台提供的Ascend计算,单卡计算,就报这个错误。求好心人帮帮孩子的课程设计。
具体报错日志上下文
===finish data synchronization===
===save flag===
Finish sync data from s3://lwjwyq/pretrain/ to /cache/checkpoint_path.
Preload downloaded: ['cspdarknet53_backbone.ckpt']
from path: /home/ma-user/modelarts/outputs/train_url_0/
to path: /cache/train
INFO:root:No files to copy.
===finish data synchronization===
===save flag===
Finish sync data from /home/ma-user/modelarts/outputs/train_url_0/ to /cache/train.
Workspace downloaded: []
Zip file path: /cache/data/coco.zip
Unzip file save dir: /cache/data
This is not zip.
===Finish extract data synchronization===
Device: 0, Finish sync unzip data from /cache/data/coco.zip to /cache/data.
Traceback (most recent call last):
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 303, in <module>
run_train()
File "/home/ma-user/modelarts/user-job-dir/code/model_utils/moxing_adapter.py", line 105, in wrapped_func
run_func(*args, **kwargs)
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 181, in run_train
profiler = set_default()
File "/home/ma-user/modelarts/user-job-dir/code/train.py", line 59, in set_default
device_id=device_id, ascend_config={"precision_mode": "allow_fp32_to_fp16"})
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/_checkparam.py", line 1090, in wrapper
return func(*args, **kwargs)
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/_checkparam.py", line 1059, in wrapper
return func(*args, **kwargs)
File "/home/ma-user/anaconda/lib/python3.7/site-packages/mindspore/context.py", line 921, in set_context
raise ValueError(f"For 'context.set_context', the keyword argument {key} is not recognized! For detailed "
ValueError: For 'context.set_context', the keyword argument ascend_config is not recognized! For detailed usage of 'set_context', please refer to the Mindspore official website.
[ModelArts Service Log]2023-12-27 17:11:28,718 - ERROR - proc-rank-0-device-0 (pid: 346) has exited with non-zero code: 1
[ModelArts Service Log]2023-12-27 17:11:28,718 - INFO - Begin destroy training processes
[ModelArts Service Log]2023-12-27 17:11:28,718 - INFO - proc-rank-0-device-0 (pid: 346) has exited before receiving the term signal
[ModelArts Service Log]2023-12-27 17:11:28,719 - INFO - End destroy training processes
time="2023-12-27T17:11:28+08:00" level=info msg="the run_train script is exiting..." Component=ShellScripts Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="the run_train 180 script is exited with 1" Component=ShellScripts Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="command is exit with 1" file="process.go:178" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="NPU training process exits with exit code 1, and the environment will be retained for 0s." file="process.go:226"
time="2023-12-27T17:11:28+08:00" level=info msg="the environment has been retained for 0s." file="process.go:228"
time="2023-12-27T17:11:28+08:00" level=info msg="DetectResult msg no error\n" file="process.go:206" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="start parse ascend log, current time is :2023-12-27 17:11:28" file="ascend_log_handler.go:129" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="ascend_fd binary found, path: /home/ma-user/modelarts/package/ascend-fd" file="executor.go:116" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
time="2023-12-27T17:11:28+08:00" level=info msg="zombie process cleaner is exiting" file="cleaner_unix.go:53" Command=bootstrap/run Component=ma-training-toolkit Platform=ModelArts-Service
The parse job starts. Please wait.
All parse job succeeded.
The parse job is complete.