Running the Upgrade Command

Before You Start

• Before using MindCluster Ascend Deployer for upgrade, ensure that the drive storage space of the MindCluster Ascend Deployer executor is greater than 16 GB.

Prerequisites

• You have downloaded required software packages.
• The upgrade user is the root user and has the execution permission on the install.sh script.

Upgrade Procedure

1. Log in to the target device as the upgrade user of the software package.
2. Run the upgrade command.
   • Run the ascend-deployer command in any directory on the local host when MindCluster Ascend Deployer is installed using pip.

     ascend-deployer --upgrade=<package_name_1>,<package_name_2>

   • Run the bash install.sh command in the ascend_deployer directory when MindCluster Ascend Deployer is installed by decompressing the downloaded ZIP package.

     bash install.sh --upgrade=<package_name_1>,<package_name_2>

   See Table 1 for the example.

   For details about the value range of <package_name_x>, see Supported Installation and Upgrade Scenarios or run the bash install.sh --help command to view all available parameters.

   Upgrade the packages in the following sequence: NPU > CANN, MindCluster (performance test, fault diagnosis, and cluster scheduling components). During the upgrade, the version of the CANN package in the resources directory must match that of the NPU.

   

   When you run the upgrade command, MindCluster Ascend Deployer checks whether the environment meets the upgrade requirements by default. If any error information is displayed, determine whether to add the --skip_check option to skip the check and continue the upgrade. Example:

   bash install.sh --upgrade=<package_name_1>,<package_name_2>  --skip_check

   Table 1 Upgrade commands

   Type	Upgrade Command
   NPU driver and firmware (either of the listed commands)	bash install.sh --upgrade=npu
   	bash install.sh --upgrade=driver,firmware
   MCU firmware	bash install.sh --upgrade=mcu
   CANN (training, inference, and development and debugging)	bash install.sh --upgrade=kernels,toolkit
   CANN (edge inference)	bash install.sh --upgrade=nnrt,kernels
   CANN (training and inference)	bash install.sh --upgrade=nnae,kernels
   MindCluster cluster scheduling	bash install.sh --upgrade=ascend-device-plugin,ascend-docker-runtime,noded,npu-exporter,volcano,ascend-operator,clusterd,resilience-controller
   MindCluster performance test	bash install.sh --upgrade=toolbox
   MindCluster fault diagnosis	bash install.sh --upgrade=fault-diag
   Description For details about the restrictions on installing the preceding software packages, see Table 1. For details about all available values of <package_name_x>, see Software Packages That Can Be Installed and Upgraded.

3. (Optional) If the CANN and MindCluster performance test component need to be upgraded, sign Huawei Enterprise End User License Agreement (EULA) to proceed to the upgrade process and enter y or Y to accept the agreement (entering any other character means a refusal). After you accept the agreement, the upgrade automatically starts.

   If the current language environment does not meet the requirements, run the following command to configure the default language environment:

   • Chinese

     export LANG=zh_CN.UTF-8

   • English

     export LANG=en_US.UTF-8

4. (Optional) If the driver and firmware need to be upgraded, you are advised to restart the system immediately after the upgrade is complete.
   For upgrade on a single server, run the reboot command. For batch upgrade, run the following command to restart all devices.

   1. If MindCluster Ascend Deployer is deployed on a target server for software upgrade, you need to mask the IP address of the local server in inventory_file, as shown in the following information in bold. Otherwise, during execution of step 4.b, the local server may have been restarted before the restart command is sent to other servers. As a result, other servers are not restarted. If MindCluster Ascend Deployer is deployed on a general-purpose server, skip this step.

      #IP address of the local server ansible_ssh_user="root" # Mask the IP address of the local server.

   2. Restart other servers.

      ansible -i inventory_file all -m shell -a 'reboot'

   3. After other servers are restarted, unmask the local IP address in inventory_file and run the reboot command to restart the local server.