This document applies only to Kubernetes-based cluster service deployment. For details about the deployment diagram, see .
Figure 1 K8s cluster deployment overview[object Object][object Object]
Based on the distribution and inference modes of Server inference service instances on cluster compute nodes (inference servers), there are two available deployment modes:
Table 1 Deployment modes
[object Object]
Cluster containerized deployment relies on Kubernetes and MindCluster. For deployment scenarios, see . For details about the Kubernetes components, see . For details about the MindCluster components, see "Introduction" > "Component Overview" in MindCluster Cluster Scheduling User Guide.
Table 2 Dependency list[object Object][object Object]
Installation method 1 (recommended):
Install the kubectl, kubeadm, and kubelet tools of Kubernetes.
[object Object]
Use kubeadm to create a Kubernetes cluster. For details about how to install kubeadm and create a Kubernetes cluster, see on the Kubernetes official website.
[object Object]
Installation method 2:
Install Kubernetes by referring to the Alibaba image source. The following operations are performed on the management node. It is recommended that the software with the Arm architecture be used.
Obtain the kubectl, kubeadm, and kubelet packages of Kubernetes. The following uses v1.25 as an example.
Install kubectl, kubeadm, and kubelet by referring to the home page of the Alibaba Kubernetes image official website.
[object Object]
Run the following command to query the dependencies and images required for deploying Kubernetes, as shown in .
[object Object]Figure 2 Query result of the required dependencies and images [object Object][object Object]
According to the query result, you need to run the
[object Object]command to install the dependencies and images in sequence. An example command is as follows:[object Object]Run the following command to initialize the Kubernetes cluster. If information similar to that shown in is displayed, the initialization is successful.
[object Object]Figure 3 Successful Kubernetes cluster initialization[object Object][object Object]
Then, execute the content displayed in , as shown in the following:
[object Object][object Object][object Object]Run the following command to check whether the statuses of default startup items are normal, as shown in . If the statuses are
[object Object], the items are normal.[object Object]Figure 4 Checking the statuses [object Object][object Object]
(Optional) If a service whose name starts with
[object Object]is not in the running state, add the network protocol framework service to the Kubernetes cluster. The Calico framework is recommended. (Skip this step if the pod status is normal.)Run the following commands to obtain the Calico-related images. (If the network is disconnected, reset the network proxy environment variables.)
[object Object][object Object]
Run the following command to download the Calico YAML file (after this step is complete, cancel the network proxy settings):
[object Object]Run the
[object Object]command to modify the file. Find the[object Object]field (in line 4444) and add the following content below it:[object Object][object Object]
Figure 5 Modified content
Start Calico.
[object Object]
Run the following command to reset Kubernetes configurations. If information similar to that shown in is displayed, the Kubernetes configurations are successfully reset.
Figure 5 Successful reset [object Object][object Object]
[object Object][object Object]
If only one server is used in the cluster, you do not need to add compute nodes. In this case, skip the following steps.
The node to be added must meet the following requirements:
kubeadm and kubelet have been installed.
On the management node, create a token and ca-cert code required for adding a new node to the cluster.
The token and ca-cert code are valid for 24 hours. If they have expired, use the following commands to create new ones.
Create a token.
[object Object]Create a ca-cert code.
[object Object]
[object Object]
Run the following command on the new node to add it to the cluster:
[object Object]Parameters:
[object Object]: IP address and port number of Kubernetes on the management node.[object Object]: token for node adding.[object Object]: hash value of the certificate for adding the node to the cluster.
Run the following command on the new node to query the host name:
[object Object]If the host name conflicts with the name of another node in the cluster, modify the
[object Object]file to change the host name.Run the
[object Object]command on the management node to view the node information. As shown in ,[object Object]is the new node.Figure 6 Adding a node [object Object][object Object]
Run the following command on the management node to add the
[object Object]or[object Object]label to the new node based on the actual NPU type.[object Object]Run the following command on the management node to check the
[object Object]label added to the new node, as shown in . If[object Object]is displayed, the operation is successful.[object Object]Figure 7 Label
[object Object][object Object][object Object]
[object Object]
Run the following command on the management node to check the container runtime used by the current cluster nodes.
[object Object]Check whether the node uses
[object Object]or[object Object]based on the[object Object]column in the command output.Modify the corresponding configuration file based on the container runtime to enable log rotation.
For Docker runtime:
Edit
[object Object]. Add or update the[object Object]field (recommended: a maximum log file size of 20 MB and a maximum of 10 files. Users can configure these values as needed):[object Object]After the modification is complete, run the following commands to restart the Docker service for the configuration to take effect:
[object Object]For Containerd runtime:
Edit the kubelet config file (typically
[object Object]). Add or modify the following parameters at the same level as[object Object]and[object Object](recommended: a maximum log file size of 20 MB and a maximum of 10 files. Users can configure these values as needed):[object Object]After the modification, run the following commands to restart the kubelet service for the configuration to take effect:
[object Object]
The cluster management components depend on the Ascend Docker Runtime, Ascend Device Plugin, Volcano, and Ascend Operator components in MindCluster. Volcano and Ascend Operator need to be installed on the management node, and other components need to be installed on compute nodes.
Create node labels, users, log directories, and namespaces by referring to in MindCluster Cluster Scheduling User Guide.
Install Ascend Docker Runtime by referring to "" > "Installing Ascend Docker Runtime in the containerd Scenario" in MindCluster Cluster Scheduling User Guide.
Install the Ascend Device Plugin using the
[object Object]file by referring to in MindCluster Cluster Scheduling User Guide.[object Object]
Install Volcano by referring to in MindCluster Cluster Scheduling User Guide.
[object Object]
Install Ascend Operator by referring to in MindCluster Cluster Scheduling User Guide.