物理机的相关信息：
1、系统openEuler 22.03 ，安装dockers，运行dockers建立容器deepseek-7b
2、进入容器docker exec -it deepseek-7b bash
查看npu-smi info
3安装ModelScope pip install modelscope
4下载完整模型 modelscope download --model deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
5修改模型路径下的config.json文件，中torch_dtype字段改为float16
6vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
修改的参数为：
npuDeviceIds：我是一张卡300iduo ，有2个芯片 ，测试 [0,1] ，可以识别到2个芯片
truncation：false
modelWeightPath：模型的路径
worldSize：2
httpsEnabled" : false
7、修改权限
8、启动服务
9、另外打开终端测试推理，
乱码！！！！！！！！！！！！！！！！！！
改成英文问题试试，回答正常：
在打开一个终端，看看npu使用情况
点赞
收藏
回复
分享
举报
已解决
物理机的相关信息：
[root@localhost ~]# uname -r 5.10.0-257.0.0.160.oe2203sp4.x86_64 [root@localhost ~]# docker -v Docker version 26.1.3, build b72abbb [root@localhost ~]# npu-smi info +--------------------------------------------------------------------------------------------------------+ | npu-smi 24.1.0.1 Version: 24.1.0.1 | +-------------------------------+-----------------+------------------------------------------------------+ | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) | | Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) | +===============================+=================+======================================================+ | 1 310P3 | OK | NA 40 0 / 0 | | 0 0 | 0000:01:00.0 | 0 1641 / 44280 | +-------------------------------+-----------------+------------------------------------------------------+ | 1 310P3 | OK | NA 40 0 / 0 | | 1 1 | 0000:01:00.0 | 0 1274 / 43693 | +===============================+=================+======================================================+ +-------------------------------+-----------------+------------------------------------------------------+ | NPU Chip | Process id | Process name | Process memory(MB) | +===============================+=================+======================================================+ | No running processes found in NPU 1 | +===============================+=================+======================================================+ [root@localhost ~]# free -h total used free shared buff/cache available Mem: 30Gi 1.2Gi 29Gi 11Mi 797Mi 29Gi Swap: 7.7Gi 0B 7.7Gi [root@localhost ~]# lscpu 架构： x86_64 CPU 运行模式： 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual 字节序： Little Endian CPU: 8 在线 CPU 列表： 0-7 厂商 ID： GenuineIntel BIOS Vendor ID: Intel 型号名称： Intel(R) Xeon(R) E-2134 CPU @ 3.50GHz BIOS Model name: Intel(R) Xeon(R) E-2134 CPU @ 3.50GHz CPU 系列： 6 型号： 158 每个核的线程数： 2 每个座的核数： 4 座： 1 步进： 10 CPU 最大 MHz： 4500.0000 CPU 最小 MHz： 800.0000 BogoMIPS： 6999.82 标记： fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd tscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx s mx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dno wprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle a vx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_act_win dow hwp_epp md_clear flush_l1d Virtualization features: 虚拟化： VT-x Caches (sum of all): L1d: 128 KiB (4 instances) L1i: 128 KiB (4 instances) L2: 1 MiB (4 instances) L3: 8 MiB (1 instance) NUMA: NUMA 节点： 1 NUMA 节点0 CPU： 0-7 Vulnerabilities: Gather data sampling: Vulnerable: No microcode Itlb multihit: KVM: Mitigation: VMX disabled L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Mds: Mitigation; Clear CPU buffers; SMT vulnerable Meltdown: Mitigation; PTI Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable Retbleed: Mitigation; IBRS Spec rstack overflow: Not affected Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Spectre v2: Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected Srbds: Vulnerable: No microcode Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable [root@localhost ~]#docker pull swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts 1.0.0-300I-Duo-py311-openeuler24.03-lts: Pulling from ascendhub/mindie 882fa63c3c86: Pull complete 39c1d5183590: Pull complete 2d3a1c83d9a5: Pull complete b5200a476f4b: Pull complete ea5cd8c1493c: Pull complete 114b3a949df0: Pull complete Digest: sha256:1fbb080e488b85f0a93dc68aa013d1d5dc775e51b9e1b6fc5fbf9638a5a530091、系统openEuler 22.03 ，安装dockers，运行dockers建立容器deepseek-7b
2、进入容器docker exec -it deepseek-7b bash
查看npu-smi info
3安装ModelScope pip install modelscope
4下载完整模型 modelscope download --model deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
5修改模型路径下的config.json文件，中torch_dtype字段改为float16
6vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
修改的参数为：
npuDeviceIds：我是一张卡300iduo ，有2个芯片 ，测试 [0,1] ，可以识别到2个芯片
truncation：falsemodelWeightPath：模型的路径
worldSize：2
httpsEnabled" : false
7、修改权限
8、启动服务
9、另外打开终端测试推理，
[root@localhost ~]# curl 127.0.0.1:1025/generate -d '{"prompt": "人为什么有思维", "max_tokens": 1000}' {"text":["人为什么有思维 (1) �-DD\n\n(1) 1/1\n</think>\n\n!中间\n\n** gun ! \n\n** - 12-21 gun **\n\n** ! 12-21- gun ! **\n\n** - 12-21- gun - **\n\n** ! 12-21- gun - ! **\n\n** - 12-21- gun - - **\n\n** ! 12-21- gun - - ! **\n\n** - 12-21- gun - - - **\n\n** ! 12-21- gun - - - ! **\n\n** - 12-21- gun - - - - **\n\n** ! 12-21- gun - - - - ! **\n\n** - 12-21- gun - - - - - **\n\n** ! 12-21- gun - - - - - ! **\n\n** - 12-21- gun - - - - - - **\n\n** ! 12-21- gun - - - - - - ! **\n\n** - 12-21- gun - - - - - - - **\n\n** ! 12-21- gun - - - - - - - ! **\n\n** - 12-21- gun - - - - - - - - **\n\n** ! 12-21- gun - - - - - - - - ! **\n\n** - 12-21- gun - - - - - - - - - **\n\n** ! 12-21- gun - - - - -"]}[root@localhost ~]#乱码！！！！！！！！！！！！！！！！！！
改成英文问题试试，回答正常：
[root@localhost ~]# curl 127.0.0.1:1025/generate -d '{"prompt": "Why do humans have thinking abilities?", "max_tokens": 1000}' {"text":["Why do humans have thinking abilities? What is the basis for the possibility of a - or the - human brain to think? I mean, the brain is just a physical organ, and the mind is just a complex of mental functions. So, what makes the human brain the seat of the mind? Why is it that the human brain is the one that thinks? Is there any particular reason? Also, when I think of the structure of the human brain and the functions of the mind, they seem to be connected. Is there any specific reason why the brain and mind are connected in such a way? And what if the brain and mind were separated? Is that even possible? I think the mind is a product of the brain and the environment, but the environment is an external factor. So, the mind and brain share a kind of 'tied' relationship, but the environment is the part that is, in a sense, free. So, the mind is a product of the environment and the brain. The environment and the brain are both factors in the origin of the mind. But the environment and the brain are the two factors. The brain and the mind share a kind of 'tied' relationship, but the environment is the part that is, in a sense, free. So, the mind is a product of the environment and the brain. The environment and the brain are the two factors. So, the environment and the brain are the two parts of the origin of the mind. But the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the environment and the brain are the two parts of the origin of the mind. So, the"]}[root@localhost ~]#在打开一个终端，看看npu使用情况