环境: 1 * 910B A2 * 8
版本: MindIE 2.3.0
镜像: swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:2.3.0-300I-Duo-py311-openeuler24.03-lts
参考:
https://www.hiascend.com/document/detail/zh/mindie/230/mindiellm/llmdev/mindie_llm0524.html
https://www.hiascend.com/document/detail/zh/mindie/230/envdeployment/instg/mindie_instg_0027.html
https://www.hiascend.com/document/detail/zh/mindie/230/mindiemotor/motordev/mindie_service0060.html
执行: 我使用MindIE Motor 在k8s 集群上部署 Controller *1, Coordinator * 1, MindIE Server(1 卡) * 运行正常 并对外提供服务。 我想增加一个pod加入到这个 MindIE 集群中,如何在不停机,不断业务的场景下扩容。
global_ranktable.json
```
'{"version": "1.0", "server_group_list": [{"group_id": "2",
"server_count": "2", "server_list": [{"server_id": "10.250.137.178", "server_ip":
"10.250.137.178", "device": [{"device_id": "4", "device_ip": "", "rank_id": "0",
"device_logical_id": "0"}]}, {"server_id": "10.250.136.42", "server_ip": "10.250.136.42",
"device": [{"device_id": "0", "device_ip": "", "rank_id": "0", "device_logical_id":
"0"}]}]}, {"group_id": "1", "server_count": "1", "server_list": [{"server_ip":
"10.250.180.26"}]}, {"group_id": "0", "server_count": "1", "server_list": [{"server_ip":
"10.250.180.56"}]}], "status": "completed"}'
```
问题:
1.如何在不改变原的 MindIE 集群 的global_ranktable.json 加入一个pod,同时不影响正在对外的业务
2.在同一个k8s 的集群中 ,部署多个 MindIE 集群 后部署的MindIE Server 的POD 总是带 其他MindIE 的环境变量 如以下 但是我的集群叫其他名字,是否有方法杜绝这个
MINDIE_MS_COORDINATOR_INFER_LLMSERVICE_SAMPLE1_PORT_1025_TCP_ADDR=10.96.168.46
MINDIE_SERVICE_LLMSERVICE_SAMPLE1_PORT_1025_TCP_PROTO=tcp
MINDIE_SERVICE_LLMSERVICE_SAMPLE1_PORT_1025_TCP=tcp://10.96.168.215:1025
MINDIE_MS_COORDINATOR_INFER_LLMSERVICE_SAMPLE1_SERVICE_PORT=1025
环境: 1 * 910B A2 * 8
版本: MindIE 2.3.0
镜像: swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:2.3.0-300I-Duo-py311-openeuler24.03-lts
参考:
https://www.hiascend.com/document/detail/zh/mindie/230/mindiellm/llmdev/mindie_llm0524.html
https://www.hiascend.com/document/detail/zh/mindie/230/envdeployment/instg/mindie_instg_0027.html
https://www.hiascend.com/document/detail/zh/mindie/230/mindiemotor/motordev/mindie_service0060.html
执行: 我使用MindIE Motor 在k8s 集群上部署 Controller *1, Coordinator * 1, MindIE Server(1 卡) * 运行正常 并对外提供服务。 我想增加一个pod加入到这个 MindIE 集群中,如何在不停机,不断业务的场景下扩容。
global_ranktable.json
```
'{"version": "1.0", "server_group_list": [{"group_id": "2",
"server_count": "2", "server_list": [{"server_id": "10.250.137.178", "server_ip":
"10.250.137.178", "device": [{"device_id": "4", "device_ip": "", "rank_id": "0",
"device_logical_id": "0"}]}, {"server_id": "10.250.136.42", "server_ip": "10.250.136.42",
"device": [{"device_id": "0", "device_ip": "", "rank_id": "0", "device_logical_id":
"0"}]}]}, {"group_id": "1", "server_count": "1", "server_list": [{"server_ip":
"10.250.180.26"}]}, {"group_id": "0", "server_count": "1", "server_list": [{"server_ip":
"10.250.180.56"}]}], "status": "completed"}'
```
问题:
1.如何在不改变原的 MindIE 集群 的global_ranktable.json 加入一个pod,同时不影响正在对外的业务
2.在同一个k8s 的集群中 ,部署多个 MindIE 集群 后部署的MindIE Server 的POD 总是带 其他MindIE 的环境变量 如以下 但是我的集群叫其他名字,是否有方法杜绝这个
MINDIE_MS_COORDINATOR_INFER_LLMSERVICE_SAMPLE1_PORT_1025_TCP_ADDR=10.96.168.46
MINDIE_SERVICE_LLMSERVICE_SAMPLE1_PORT_1025_TCP_PROTO=tcp
MINDIE_SERVICE_LLMSERVICE_SAMPLE1_PORT_1025_TCP=tcp://10.96.168.215:1025
MINDIE_MS_COORDINATOR_INFER_LLMSERVICE_SAMPLE1_SERVICE_PORT=1025