使用2.0.RC1-300I-Duo-py311-openeuler24.03-lts镜像运行不起来之前1.0版本可以运行的模型
收藏回复举报
使用2.0.RC1-300I-Duo-py311-openeuler24.03-lts镜像运行不起来之前1.0版本可以运行的模型
t('forum.solved') 已解决
发表于2025-05-09 10:01:07
0 查看

新下载的2.0.RC1-300I-Duo-py311-openeuler24.03-lts镜像,本来想运行qwen3,结果发现这个镜像似乎还不支持。所以想着运行一下老的模型(qwq-32b),不过发现也不能运行起来。config的参数优化或者不优化都不行,查看debug日志发现:

[2025-05-09 09:52:36.618+08:00] [4258] [4260] [batchscheduler] [INFO] [model_backend.cpp:132] : [model_backend] {asyncBatchscheduler, false},{async_infer, false},{backend_bin_path, /usr/local/Ascend/mindie/2.0.RC1/mindie-llm/bin/},{backend_log_file, /usr/local/Ascend/mindie/2.0.RC1/mindie-service/~/mindie/log/},{backend_modelInstance_id, 0},{backend_type, atb},{block_size, 128},{cpu_mem, 5},{deploy_type, INTER_PROCESS},{distributed_enable, false},{executor_type, LLM_EXECUTOR_PYTHON},{globalRankIds, },{globalWorldSize, 0},{infer_mode, standard},{interNodeKmcKsfMaster, tools/pmt/master/ksfa},{interNodeKmcKsfStandby, tools/pmt/standby/ksfb},{interNodeTLSEnabled, 1},{interNodeTlsCaFiles, ca.pem,},{interNodeTlsCaPath, security/grpc/ca/},{interNodeTlsCert, security/grpc/certs/server.pem},{interNodeTlsCrlFiles, server_crl.pem,},{interNodeTlsCrlPath, security/grpc/certs/},{interNodeTlsPk, security/grpc/keys/server.key.pem},{interNodeTlsPkPwd, security/grpc/pass/mindie_server_key_pwd.txt},{isMaster, 0},{is_dmi_infer, 0},{localIP, },{log_error, 1},{log_file_num, 20},{log_file_size, 20},{log_info, 1},{log_verbose, 0},{log_warning, 1},{masterIP, },{max_batch_size, 200},{max_input_len, 2048},{max_iter_times, 512},{max_prefill_tokens, 8192},{max_seq_len, 2560},{model_id, /model/QwQ-32B},{model_instance_number, 1},{model_instance_type, Standard},{model_name, qwq-32b},{multiNodesInferEnabled, 0},{multiNodesInferPort, 1120},{npu_device_ids, },{npu_mem, -1},{slaveIPs, },{speculation_gamma, 0},{threadNum, 1},{trust_remote_code, 0},{world_size, 0},

[2025-05-09 09:52:36.618+08:00] [4258] [4260] [batchscheduler] [ERROR] [model_backend.cpp:192] : [MIE04E120409] [model_backend] Value of modelConfigKey 'world_size' cannot be smaller than 1

[2025-05-09 09:52:36.618+08:00] [4258] [4260] [batchscheduler] [ERROR] [model_backend.cpp:324] : [MIE04E12041C] [model_backend] InstanceConfigInit failed

很奇怪这里为什么会认为world_size是0?因为我的配置中很明确写的是4(默认值也是4)

我要发帖子