我用onnx转成了一个om模型,并用如下命令测试推理速度:
HwHiAiUser@Euler:~/Deploy/yolov8_obb$ python3 -m ais_bench --model=v8_obb-512-640.om --loop=1000 --batchsize=1
[INFO] acl init success
[INFO] open device 0 success
[INFO] load model v8_obb-512-640.om success
[INFO] create model description success
[INFO] warm up 1 done
Inference array Processing: 0%| | 0/1 [00:00<?, ?it/s]
loop inference exec: (1000/1000)ython3 -m ais_bench --model=v8_obb-512-640.om --loop=1000 --batchsize=1
Inference array Processing: 100%|███████████████████████████████████████████| 1/1 [00:09<00:00, 9.19s/it]
[INFO] -----------------Performance Summary------------------
[INFO] NPU_compute_time (ms): min = 8.619999885559082, max = 9.017999649047852, mean = 8.686402983665467, median = 8.675999641418457, percentile(99%) = 8.91700982093811
[INFO] throughput 1000*batchsize.mean(1)/NPU_compute_time.mean(8.686402983665467): 115.1224507866457
但在模型推理中,以下代码却耗时25ms左右,请问是什么导致了这个差异?
output = model.execute([img_pre])
HwHiAiUser@Euler:~/Deploy/yolov8_obb$ python3 -m ais_bench --model=v8_obb-512-640.om --loop=1000 --batchsize=1
[INFO] acl init success
[INFO] open device 0 success
[INFO] load model v8_obb-512-640.om success
[INFO] create model description success
[INFO] warm up 1 done
Inference array Processing: 0%| | 0/1 [00:00<?, ?it/s]
loop inference exec: (1000/1000)ython3 -m ais_bench --model=v8_obb-512-640.om --loop=1000 --batchsize=1
Inference array Processing: 100%|███████████████████████████████████████████| 1/1 [00:09<00:00, 9.19s/it]
[INFO] -----------------Performance Summary------------------
[INFO] NPU_compute_time (ms): min = 8.619999885559082, max = 9.017999649047852, mean = 8.686402983665467, median = 8.675999641418457, percentile(99%) = 8.91700982093811
[INFO] throughput 1000*batchsize.mean(1)/NPU_compute_time.mean(8.686402983665467): 115.1224507866457
output = model.execute([img_pre])但在模型推理中,以下代码却耗时25ms左右,请问是什么导致了这个差异?