~# atc --help
ATC start working now, please wait for a moment.
atc.bin: usage: ./atc <args>
generate offline model example:
./atc --model=./alexnet.prototxt --weight=./alexnet.caffemodel
--framework=0 --output=./domi
generate offline model for single op example:
./atc --singleop=./op_list.json --output=./op_model
===== Basic Functionality =====
[General]
--h/help Show this help message
--mode Run mode. 0(default): generate offline model; 1: convert model to JSON format; 3: only pre-check; 5: convert ge dump txt file to JSON format; 6: display model info
[Input]
--model Model file
--weight Weight file. Required when framework is Caffe
--om The model file to be converted to json
--framework Framework type. 0:Caffe; 1:MindSpore; 3:Tensorflow; 5:Onnx
--input_format Format of input data. E.g.: "NCHW"
--input_shape Shape of input data. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument.
E.g.: "input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2"
--input_shape_range Shape range of input data. Separate multiple nodes with semicolons (;).Use double quotation marks (") to enclose each argument.
E.g.: "input_name1:[n1~n2,c1,h1,w1];input_name2:[n2,c2~c3,h2,w2]"
--dynamic_batch_size Set dynamic batch size. E.g.: "batchsize1,batchsize2,batchsize3"
--dynamic_image_size Set dynamic image size. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument.
E.g.: "imagesize1_height,imagesize1_width;imagesize2_height,imagesize2_width"
--dynamic_dims Set dynamic dims. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument.
E.g.: "dims1_n1,dims1_n2;dims2_n1,dims2_n2"
--singleop Single op definition file. atc will generate offline model(s) for single op if --singleop is set.
[Output]
--output Output file path&name(needn't suffix, will add .om automatically).
If --singleop is set, this arg specifies the directory to which the single op offline model will be generated
--output_type Set net output type. Support FP32, FP16, UINT8. E.g.: FP16, indicates that all out nodes are set to FP16.
"node1:0:FP16;node2:1:FP32", indicates setting the datatype of multiple out nodes.
--check_report The pre-checking report file. Default value is: "check_result.json"
--json The output json file path&name which is converted from a model
[Target]
--soc_version The soc version.
--core_type Set core type AiCore or VectorCore. VectorCore: use vector core. Default value is: AiCore
--aicore_num Set aicore num
===== Advanced Functionality =====
[Feature]
--out_nodes Output nodes designated by users. Separate multiple nodes with semicolons (;).Use double quotation marks (") to enclose each argument.
E.g.: "node_name1:0;node_name1:1;node_name2:0"
--input_fp16_nodes Input node datatype is fp16. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument. E.g.: "node_name1;node_name2"
--insert_op_conf Config file to insert new op
--op_name_map Custom op name mapping file
Note: A semicolon(;) cannot be included in each path, otherwise the resolved path will not match the expected one.
--is_input_adjust_hw_layout Intput node datatype is fp16 and format is NC1HWC0, used with input_fp16_nodes. E.g.: "true,true,false,true"
--is_output_adjust_hw_layout Net output node datatype is fp16 and format is NC1HWC0, used with out_nodes. E.g.: "true,true,false,true"
[Model Tuning]
--disable_reuse_memory The switch of reuse memory. Default value is : 0. 0 means reuse memory, 1 means do not reuse memory.
--fusion_switch_file Set fusion switch file path
--enable_scope_fusion_passes validate the non-general scope fusion passes, multiple names can be set and separated by ','. E.g.: ScopePass1,ScopePass2,...
--enable_single_stream Enable single stream. true: enable; false(default): disable
--enable_small_channel Set enable small channel. 0(default): disable; 1: enable
--enable_compress_weight Enable compress weight. true: enable; false(default): disable
--compress_weight_conf Config file to compress weight
--compression_optimize_conf Config file to compress optimize
--sparsity Optional; enable structured sparse. 0(default): disable; 1: enable
--buffer_optimize Set buffer optimize. Support "l2_optimize" (default), "l1_optimize", "off_optimize"
--mdl_bank_path Set the path of the custom repository generated after model tuning.
[Operator Tuning]
--op_precision_mode Set the path of operator precision mode configuration file (.ini)
--precision_mode precision mode, support force_fp16(default), force_fp32, allow_mix_precision, allow_fp32_to_fp16, must_keep_origin_dtype.
--modify_mixlist Set the path of operator mixed precision configuration file.
--keep_dtype Retains the precision of certain operators in inference scenarios by using a configuration file.
--customize_dtypes Set the path of custom dtypes configuration file.
--auto_tune_mode Set tune mode. E.g.: "GA,RL", support configure multiple, spit by ,
--op_bank_path Set the path of the custom repository generated after operator tuning with Auto
Tune.
--op_select_implmode Set op select implmode. Support high_precision, high_performance, high_precision_for_all, high_performance_for_all. default: high_performance
--optypelist_for_implmode Appoint which op to select implmode, cooperated with op_select_implmode.
Separate multiple nodes with commas (,). Use double quotation marks (") to enclose each argument. E.g.: "node_name1,node_name2"
--op_debug_level Debug enable for TBE operator building.
0 (default): Disable debug; 1: Enable TBE pipe_all, and generate the operator CCE file and Python-CCE mapping file (.json);
2: Enable TBE pipe_all, generate the operator CCE file and Python-CCE mapping file (.json), and enable the CCE compiler -O0-g.
3: Disable debug, and keep generating kernel file (.o and .json)
4: Disable debug, keep generation kernel file (.o and .json) and generate the operator CCE file (.cce) and the UB fusion computing description file (.json)
[Debug]
--save_original_model Control whether to output original model. E.g.: true: output original model
--log Generate log with level. Support debug, info, warning, error, null
--dump_mode The switch of dump json with shape, to be used with mode 1. 0(default): disable; 1: enable.
--debug_dir Set the save path of operator compilation intermediate files.
Default value: ./kernel_meta
--status_check switch for op status check such as overflow.
false: disable; true: enable.
--op_compiler_cache_dir Set the save path of operator compilation cache files.
Default value: $HOME/atc_data
--op_compiler_cache_mode Set the operator compilation cache mode.Options are disable(default), enable and force(force to refresh the cache)
--display_model_info enable for display model info; 0(default): close display, 1: open display.
--shape_generalized_build_mode For selecting the mode of shape generalization when build graph.
shape_generalized: Shape will be generalized during graph build
shape_precise: Shape will not be generalized, use precise shape.
No modules matched: use -help
点赞
收藏
回复
分享
举报
已解决
Atlas200: RuntimeError: Initialize:/usr1/workspace/FPTA_Daily_v1.8.1_py38/CODE/torch_npu/csrc/core/npu/sys_ctrl/npu_sys_ctrl.cpp:132 NPU error, error code is 507033
[Log]
[SYSTEM INFO]
1. demo.py
import torch import torch_npu import torchvision print(torch.__version__) print(torch_npu.__version__) print(torchvision.__version__) print(torch.npu.is_available()) device = torch.device('npu:0') # device = torch.device('cpu:0') # cpu正常输出结果 a = torch.rand(3, 30).to(device=device) b = torch.rand(30, 3).to(device=device) print(a @ b)[ERROR]
1.8.1 1.8.1.post2 0.9.1 True Traceback (most recent call last): File "/home/USERNAME/cvyolo/de/demo.py", line 17, in <module> a = torch.rand(3, 30).to(device=device) File "/home/USERNAME/pyenvlist/envyolo/lib/python3.8/site-packages/torch_npu/utils/device_guard.py", line 45, in wrapper return func(*args, **kwargs) File "/home/USERNAME/pyenvlist/envyolo/lib/python3.8/site-packages/torch_npu/utils/tensor_methods.py", line 88, in _to return torch_npu._C.to(self, *args, **kwargs) RuntimeError: Initialize:/usr1/workspace/FPTA_Daily_v1.8.1_py38/CODE/torch_npu/csrc/core/npu/sys_ctrl/npu_sys_ctrl.cpp:132 NPU error, error code is 507033 [Error]: Failed to start the device. Rectify the fault based on the error information in the log, or you can ask us at follwing gitee link by issues: https://gitee.com/ascend/pytorch/issue E39999: Inner Error! E39999 TsdOpen failed. devId=0, tdt error=35[FUNC:startAicpuExecutor][FILE:runtime.cc][LINE:1559] Start aicpu executor failed, retCode=0x7020009 devId=0[FUNC:DeviceRetain][FILE:runtime.cc][LINE:1990] Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:1843] Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:1870] Check param failed, context can not be null.[FUNC:NewDevice][FILE:api_impl.cc][LINE:1244] new device failed, retCode=0x7010006[FUNC:SetDevice][FILE:api_impl.cc][LINE:1265] rtSetDevice execute failed, reason=[device retain error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:45] open device 0 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162] ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:3365] rtGetDevMsg execute failed, reason=[context pointer null][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:45]2. Environment
2.1 python
2.1 driver
2.3 firmware
~# /usr/local/Ascend/driver/tools/upgrade-tool --device_index -1 --component -1 --version { Get component version(1.80.22.2.220) succeed for deviceId(0), componentType(0). {"device_id":0, "component":nve, "version":1.80.22.2.220} Get component version(1.80.22.2.220) succeed for deviceId(0), componentType(1). {"device_id":0, "component":xloader, "version":1.80.22.2.220} Get component version(1.3.2.0.893) succeed for deviceId(0), componentType(2). {"device_id":0, "component":m3fw, "version":1.3.2.0.893} Get component version(1.80.22.2.220) succeed for deviceId(0), componentType(3). {"device_id":0, "component":uefi, "version":1.80.22.2.220} Get component version(1.3.2.0.893) succeed for deviceId(0), componentType(4). {"device_id":0, "component":tee, "version":1.3.2.0.893} Get component version(1.3.2.0.893) succeed for deviceId(0), componentType(5). {"device_id":0, "component":kernel, "version":1.3.2.0.893} Get component version(1.3.2.0.893) succeed for deviceId(0), componentType(6). {"device_id":0, "component":dtb, "version":1.3.2.0.893} }2.4 CANN-toolkit
2.5 nnae & atc
ATC命令能够正常运行
ATC --help 输出
~# atc --help ATC start working now, please wait for a moment. atc.bin: usage: ./atc <args> generate offline model example: ./atc --model=./alexnet.prototxt --weight=./alexnet.caffemodel --framework=0 --output=./domi generate offline model for single op example: ./atc --singleop=./op_list.json --output=./op_model ===== Basic Functionality ===== [General] --h/help Show this help message --mode Run mode. 0(default): generate offline model; 1: convert model to JSON format; 3: only pre-check; 5: convert ge dump txt file to JSON format; 6: display model info [Input] --model Model file --weight Weight file. Required when framework is Caffe --om The model file to be converted to json --framework Framework type. 0:Caffe; 1:MindSpore; 3:Tensorflow; 5:Onnx --input_format Format of input data. E.g.: "NCHW" --input_shape Shape of input data. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument. E.g.: "input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2" --input_shape_range Shape range of input data. Separate multiple nodes with semicolons (;).Use double quotation marks (") to enclose each argument. E.g.: "input_name1:[n1~n2,c1,h1,w1];input_name2:[n2,c2~c3,h2,w2]" --dynamic_batch_size Set dynamic batch size. E.g.: "batchsize1,batchsize2,batchsize3" --dynamic_image_size Set dynamic image size. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument. E.g.: "imagesize1_height,imagesize1_width;imagesize2_height,imagesize2_width" --dynamic_dims Set dynamic dims. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument. E.g.: "dims1_n1,dims1_n2;dims2_n1,dims2_n2" --singleop Single op definition file. atc will generate offline model(s) for single op if --singleop is set. [Output] --output Output file path&name(needn't suffix, will add .om automatically). If --singleop is set, this arg specifies the directory to which the single op offline model will be generated --output_type Set net output type. Support FP32, FP16, UINT8. E.g.: FP16, indicates that all out nodes are set to FP16. "node1:0:FP16;node2:1:FP32", indicates setting the datatype of multiple out nodes. --check_report The pre-checking report file. Default value is: "check_result.json" --json The output json file path&name which is converted from a model [Target] --soc_version The soc version. --core_type Set core type AiCore or VectorCore. VectorCore: use vector core. Default value is: AiCore --aicore_num Set aicore num ===== Advanced Functionality ===== [Feature] --out_nodes Output nodes designated by users. Separate multiple nodes with semicolons (;).Use double quotation marks (") to enclose each argument. E.g.: "node_name1:0;node_name1:1;node_name2:0" --input_fp16_nodes Input node datatype is fp16. Separate multiple nodes with semicolons (;). Use double quotation marks (") to enclose each argument. E.g.: "node_name1;node_name2" --insert_op_conf Config file to insert new op --op_name_map Custom op name mapping file Note: A semicolon(;) cannot be included in each path, otherwise the resolved path will not match the expected one. --is_input_adjust_hw_layout Intput node datatype is fp16 and format is NC1HWC0, used with input_fp16_nodes. E.g.: "true,true,false,true" --is_output_adjust_hw_layout Net output node datatype is fp16 and format is NC1HWC0, used with out_nodes. E.g.: "true,true,false,true" [Model Tuning] --disable_reuse_memory The switch of reuse memory. Default value is : 0. 0 means reuse memory, 1 means do not reuse memory. --fusion_switch_file Set fusion switch file path --enable_scope_fusion_passes validate the non-general scope fusion passes, multiple names can be set and separated by ','. E.g.: ScopePass1,ScopePass2,... --enable_single_stream Enable single stream. true: enable; false(default): disable --enable_small_channel Set enable small channel. 0(default): disable; 1: enable --enable_compress_weight Enable compress weight. true: enable; false(default): disable --compress_weight_conf Config file to compress weight --compression_optimize_conf Config file to compress optimize --sparsity Optional; enable structured sparse. 0(default): disable; 1: enable --buffer_optimize Set buffer optimize. Support "l2_optimize" (default), "l1_optimize", "off_optimize" --mdl_bank_path Set the path of the custom repository generated after model tuning. [Operator Tuning] --op_precision_mode Set the path of operator precision mode configuration file (.ini) --precision_mode precision mode, support force_fp16(default), force_fp32, allow_mix_precision, allow_fp32_to_fp16, must_keep_origin_dtype. --modify_mixlist Set the path of operator mixed precision configuration file. --keep_dtype Retains the precision of certain operators in inference scenarios by using a configuration file. --customize_dtypes Set the path of custom dtypes configuration file. --auto_tune_mode Set tune mode. E.g.: "GA,RL", support configure multiple, spit by , --op_bank_path Set the path of the custom repository generated after operator tuning with Auto Tune. --op_select_implmode Set op select implmode. Support high_precision, high_performance, high_precision_for_all, high_performance_for_all. default: high_performance --optypelist_for_implmode Appoint which op to select implmode, cooperated with op_select_implmode. Separate multiple nodes with commas (,). Use double quotation marks (") to enclose each argument. E.g.: "node_name1,node_name2" --op_debug_level Debug enable for TBE operator building. 0 (default): Disable debug; 1: Enable TBE pipe_all, and generate the operator CCE file and Python-CCE mapping file (.json); 2: Enable TBE pipe_all, generate the operator CCE file and Python-CCE mapping file (.json), and enable the CCE compiler -O0-g. 3: Disable debug, and keep generating kernel file (.o and .json) 4: Disable debug, keep generation kernel file (.o and .json) and generate the operator CCE file (.cce) and the UB fusion computing description file (.json) [Debug] --save_original_model Control whether to output original model. E.g.: true: output original model --log Generate log with level. Support debug, info, warning, error, null --dump_mode The switch of dump json with shape, to be used with mode 1. 0(default): disable; 1: enable. --debug_dir Set the save path of operator compilation intermediate files. Default value: ./kernel_meta --status_check switch for op status check such as overflow. false: disable; true: enable. --op_compiler_cache_dir Set the save path of operator compilation cache files. Default value: $HOME/atc_data --op_compiler_cache_mode Set the operator compilation cache mode.Options are disable(default), enable and force(force to refresh the cache) --display_model_info enable for display model info; 0(default): close display, 1: open display. --shape_generalized_build_mode For selecting the mode of shape generalization when build graph. shape_generalized: Shape will be generalized during graph build shape_precise: Shape will not be generalized, use precise shape. No modules matched: use -help3. file.run