arm+910a 8卡环境
使用的镜像：swr.cn-central-221.ovaijisuan.com/wh-aicc-fae/mindie:910a-ascend_23.0.0-cann_8.0.rc3-py_3.10-ubuntu_22.04-aarch64-mindie_1.0.t65
http://mirrors.cn-central-221.ovaijisuan.com/detail/149.html
执行Qwen2-VL-7B-Instruct报错不支持，但https://www.hiascend.com/document/detail/zh/mindie/100/whatismindie/mindie_what_0004.html线上貌似支持，是不支持910A卡吗？
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend# cat /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
{
"Version": "1.0.0",
"LogConfig" :
{
"logLevel" : "Info",
"logFileSize" : 20,
"logFileNum" : 20,
"logPath" : "logs/mindservice.log"
},
"ServerConfig" :
{
"ipAddress" : "192.168.60.10",
"managementIpAddress": "127.0.0.2",
"port" : 1025,
"managementPort" : 1026,
"metricsPort" : 1027,
"maxLinkNum" : 1000,
"httpsEnabled" : false,
"fullTextEnabled" : false,
"tlsCaPath" : "security/ca/",
"tlsCaFile" : ["ca.pem"],
"tlsCert" : "security/certs/server.pem",
"tlsPk" : "security/keys/server.key.pem",
"tlsPkPwd" : "security/pass/key_pwd.txt",
"tlsCrl" : "security/certs/server_crl.pem",
"managementTlsCaFile" : ["management_ca.pem"],
"managementTlsCert" : "security/certs/management/server.pem",
"managementTlsPk" : "security/keys/management/server.key.pem",
"managementTlsPkPwd" : "security/pass/management/key_pwd.txt",
"managementTlsCrl" : "security/certs/management/server_crl.pem",
"kmcKsfMaster" : "tools/pmt/master/ksfa",
"kmcKsfStandby" : "tools/pmt/standby/ksfb",
"inferMode" : "standard",
"interCommTLSEnabled": false,
"interCommPort": 1121,
"interCommTlsCaFile" : "security/grpc/ca/ca.pem",
"interCommTlsCert" : "security/grpc/certs/server.pem",
"interCommPk" : "security/grpc/keys/server.key.pem",
"interCommPkPwd" : "security/grpc/pass/key_pwd.txt",
"interCommTlsCrl" : "security/certs/server_crl.pem"
},
"BackendConfig": {
"backendName" : "mindieservice_llm_engine",
"modelInstanceNumber" : 1,
"npuDeviceIds" : [[0,1,2,3,4,5,6,7]],
"tokenizerProcessNumber" : 8,
"multiNodesInferEnabled": false,
"multiNodesInferPort": 1120,
"interNodeTLSEnabled": true,
"interNodeTlsCaFile": "security/grpc/ca/ca.pem",
"interNodeTlsCert": "security/grpc/certs/server.pem",
"interNodeTlsPk": "security/grpc/keys/server.key.pem",
"interNodeTlsPkPwd": "security/grpc/pass/mindie_server_key_pwd.txt",
"interNodeTlsCrl" : "security/grpc/certs/server_crl.pem",
"interNodeKmcKsfMaster": "tools/pmt/master/ksfa",
"interNodeKmcKsfStandby": "tools/pmt/standby/ksfb",
"ModelDeployConfig":
{
"maxSeqLen" : 32768,
"maxInputTokenLen" : 8192,
"truncation" : false,
"ModelConfig" : [
{
"modelInstanceType": "Standard",
"modelName" : "qwen",
"modelWeightPath" : "/models/Qwen/Qwen2-VL-7B-Instruct",
"worldSize" : 8,
"cpuMemSize" : 5,
"npuMemSize" : -1,
"backendType": "atb"
}
]
},
"ScheduleConfig":
{
"templateType": "Standard",
"templateName" : "Standard_LLM",
"cacheBlockSize" : 128,
"maxPrefillBatchSize" : 50,
"maxPrefillTokens" : 8192,
"prefillTimeMsPerReq" : 150,
"prefillPolicyType" : 0,
"decodeTimeMsPerReq" : 50,
"decodePolicyType" : 0,
"maxBatchSize" : 200,
"maxIterTimes" : 512,
"maxPreemptCount" : 0,
"supportSelectBatch" : false,
"maxQueueDelayMicroseconds" : 5000
}
}
}
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# vi /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
vi: /root/miniconda3/envs/MindIE_1.0.T65/lib/libtinfo.so.6: no version information available (required by vi)
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# /usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon
2025-03-09 19:41:39.121297 1 [EndPoint dmi_role.cpp:237] [DmiRole::RetryThread] start retry thread.
Check path: config.json failed, by: Check Other group permission failed: Current permission is 4, but required no greater than 0.
Failed check model config.json! take modelConfig.modelWeightPath to empty
ERR: Failed to init endpoint! Please check the service log or console output.
Killed
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# chmod -R 750 /models/Qwen/
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# /usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon
2025-03-09 19:42:08.842433 1 [EndPoint dmi_role.cpp:237] [DmiRole::RetryThread] start retry thread.
2025-03-09 19:42:08.888717 108599 Log started at [warning] level
2025-03-09 19:42:08.888729 108599 Log default format: yyyy-mm-dd hh:mm:ss.uuuuuu threadid loglevel msg
2025-03-09 19:42:12.419082 108599 error [model_deploy_config.cpp:185] Failed to get vocab size from tokenizer wrapper with exception: NotImplementedError: unsupported model type: qwen2_vl；请确认atb_llm.models路径下是否存在名为qwen2_vl的文件夹。
At:
/usr/local/Ascend/atb-models/atb_llm/models/__init__.py(60): get_model
/usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(7): __init__
/root/miniconda3/envs/MindIE_1.0.T65/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__
/root/miniconda3/envs/MindIE_1.0.T65/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper
Aborted (core dumped)
点赞
收藏
回复
分享
举报
已解决
arm+910a 8卡环境
使用的镜像：swr.cn-central-221.ovaijisuan.com/wh-aicc-fae/mindie:910a-ascend_23.0.0-cann_8.0.rc3-py_3.10-ubuntu_22.04-aarch64-mindie_1.0.t65
http://mirrors.cn-central-221.ovaijisuan.com/detail/149.html
执行Qwen2-VL-7B-Instruct报错不支持，但https://www.hiascend.com/document/detail/zh/mindie/100/whatismindie/mindie_what_0004.html线上貌似支持，是不支持910A卡吗？
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend# cat /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
{
"Version": "1.0.0",
"LogConfig" :
{
"logLevel" : "Info",
"logFileSize" : 20,
"logFileNum" : 20,
"logPath" : "logs/mindservice.log"
},
"ServerConfig" :
{
"ipAddress" : "192.168.60.10",
"managementIpAddress": "127.0.0.2",
"port" : 1025,
"managementPort" : 1026,
"metricsPort" : 1027,
"maxLinkNum" : 1000,
"httpsEnabled" : false,
"fullTextEnabled" : false,
"tlsCaPath" : "security/ca/",
"tlsCaFile" : ["ca.pem"],
"tlsCert" : "security/certs/server.pem",
"tlsPk" : "security/keys/server.key.pem",
"tlsPkPwd" : "security/pass/key_pwd.txt",
"tlsCrl" : "security/certs/server_crl.pem",
"managementTlsCaFile" : ["management_ca.pem"],
"managementTlsCert" : "security/certs/management/server.pem",
"managementTlsPk" : "security/keys/management/server.key.pem",
"managementTlsPkPwd" : "security/pass/management/key_pwd.txt",
"managementTlsCrl" : "security/certs/management/server_crl.pem",
"kmcKsfMaster" : "tools/pmt/master/ksfa",
"kmcKsfStandby" : "tools/pmt/standby/ksfb",
"inferMode" : "standard",
"interCommTLSEnabled": false,
"interCommPort": 1121,
"interCommTlsCaFile" : "security/grpc/ca/ca.pem",
"interCommTlsCert" : "security/grpc/certs/server.pem",
"interCommPk" : "security/grpc/keys/server.key.pem",
"interCommPkPwd" : "security/grpc/pass/key_pwd.txt",
"interCommTlsCrl" : "security/certs/server_crl.pem"
},
"BackendConfig": {
"backendName" : "mindieservice_llm_engine",
"modelInstanceNumber" : 1,
"npuDeviceIds" : [[0,1,2,3,4,5,6,7]],
"tokenizerProcessNumber" : 8,
"multiNodesInferEnabled": false,
"multiNodesInferPort": 1120,
"interNodeTLSEnabled": true,
"interNodeTlsCaFile": "security/grpc/ca/ca.pem",
"interNodeTlsCert": "security/grpc/certs/server.pem",
"interNodeTlsPk": "security/grpc/keys/server.key.pem",
"interNodeTlsPkPwd": "security/grpc/pass/mindie_server_key_pwd.txt",
"interNodeTlsCrl" : "security/grpc/certs/server_crl.pem",
"interNodeKmcKsfMaster": "tools/pmt/master/ksfa",
"interNodeKmcKsfStandby": "tools/pmt/standby/ksfb",
"ModelDeployConfig":
{
"maxSeqLen" : 32768,
"maxInputTokenLen" : 8192,
"truncation" : false,
"ModelConfig" : [
{
"modelInstanceType": "Standard",
"modelName" : "qwen",
"modelWeightPath" : "/models/Qwen/Qwen2-VL-7B-Instruct",
"worldSize" : 8,
"cpuMemSize" : 5,
"npuMemSize" : -1,
"backendType": "atb"
}
]
},
"ScheduleConfig":
{
"templateType": "Standard",
"templateName" : "Standard_LLM",
"cacheBlockSize" : 128,
"maxPrefillBatchSize" : 50,
"maxPrefillTokens" : 8192,
"prefillTimeMsPerReq" : 150,
"prefillPolicyType" : 0,
"decodeTimeMsPerReq" : 50,
"decodePolicyType" : 0,
"maxBatchSize" : 200,
"maxIterTimes" : 512,
"maxPreemptCount" : 0,
"supportSelectBatch" : false,
"maxQueueDelayMicroseconds" : 5000
}
}
}
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# vi /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
vi: /root/miniconda3/envs/MindIE_1.0.T65/lib/libtinfo.so.6: no version information available (required by vi)
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# /usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon
2025-03-09 19:41:39.121297 1 [EndPoint dmi_role.cpp:237] [DmiRole::RetryThread] start retry thread.
Check path: config.json failed, by: Check Other group permission failed: Current permission is 4, but required no greater than 0.
Failed check model config.json! take modelConfig.modelWeightPath to empty
ERR: Failed to init endpoint! Please check the service log or console output.
Killed
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# chmod -R 750 /models/Qwen/
(MindIE_1.0.T65) root@huawei:/usr/local/Ascend/mindie/1.0.T65/mindie-llm/logs# /usr/local/Ascend/mindie/latest/mindie-service/bin/mindieservice_daemon
2025-03-09 19:42:08.842433 1 [EndPoint dmi_role.cpp:237] [DmiRole::RetryThread] start retry thread.
2025-03-09 19:42:08.888717 108599 Log started at [warning] level
2025-03-09 19:42:08.888729 108599 Log default format: yyyy-mm-dd hh:mm:ss.uuuuuu threadid loglevel msg
2025-03-09 19:42:12.419082 108599 error [model_deploy_config.cpp:185] Failed to get vocab size from tokenizer wrapper with exception: NotImplementedError: unsupported model type: qwen2_vl；请确认atb_llm.models路径下是否存在名为qwen2_vl的文件夹。
At:
/usr/local/Ascend/atb-models/atb_llm/models/__init__.py(60): get_model
/usr/local/Ascend/atb-models/atb_llm/runner/tokenizer_wrapper.py(7): __init__
/root/miniconda3/envs/MindIE_1.0.T65/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_tokenizer_wrapper.py(8): __init__
/root/miniconda3/envs/MindIE_1.0.T65/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/__init__.py(28): get_tokenizer_wrapper
Aborted (core dumped)