Cluster Scheduling Components with Containerd

Enabling Ascend Docker Runtime for Containerd

To use the Ascend Docker Runtime on the Containerd client, perform the following steps:

1. Modify the configuration file.
   Run the following commands to view the configuration file:

   mkdir /etc/containerd
   containerd config default > /etc/containerd/config.toml
   vim /etc/containerd/config.toml

   Change the runtime installation path as required. The following content in bold is an example:

   ...
    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming] 
      tls_cert_file = "" 
      tls_key_file = "" 
    [plugins."io.containerd.internal.v1.opt"] 
      path = "/opt/containerd" 
    [plugins."io.containerd.internal.v1.restart"] 
      interval = "10s" 
    [plugins."io.containerd.metadata.v1.bolt"] 
      content_sharing_policy = "shared" 
    [plugins."io.containerd.monitor.v1.cgroups"] 
      no_prometheus = false 
    [plugins."io.containerd.runtime.v1.linux"] 
      shim = "containerd-shim" 
      runtime = "/usr/local/Ascend/Ascend-Docker-Runtime/ascend-docker-runtime" 
      runtime_root = "" 
      no_shim = false 
      shim_debug = false 
    [plugins."io.containerd.runtime.v2.task"] 
      platforms = ["linux/amd64"] 
    [plugins."io.containerd.service.v1.diff-service"] 
      default = ["walking"] 
    [plugins."io.containerd.snapshotter.v1.devmapper"] 
      root_path = "" 
      pool_name = "" 
      base_image_size = "" 
      async_remove = false

2. Run the following command to restart containerd:

   systemctl restart containerd

3. Start the container.
   The following are examples of commands for starting containers.

   • Example 1: Mount a processor whose physical processor ID is 0 when starting a container.

     ctr run --runtime io.containerd.runtime.v1.linux -t --env ASCEND_VISIBLE_DEVICES=0 image-name:tag containerID

   • Example 2: Mount only NPU and management devices when starting a container. Do not mount driver-related directories.

     ctr run --runtime io.containerd.runtime.v1.linux -t --env ASCEND_VISIBLE_DEVICES=0 --env ASCEND_RUNTIME_OPTIONS=NODRV image-name:tag containerID

   • Example 3: Mount a processor whose physical processor ID is 0 and read the mounted content of the custom configuration file hostlog when starting a container.

     ctr run --runtime io.containerd.runtime.v1.linux -t --env ASCEND_VISIBLE_DEVICES=0 --env ASCEND_RUNTIME_MOUNTS=hostlog image-name:tag containerID

   • Example 4: Mount a processor whose virtual processor ID is 100 when starting a container.

     ctr run --runtime io.containerd.runtime.v1.linux -t --env ASCEND_VISIBLE_DEVICES=100 --env ASCEND_RUNTIME_OPTIONS=VIRTUAL image-name:tag containerID

   • Example 5: Split four AI Cores from the processor whose physical processor ID is 0 as virtual devices and mount them to a container when starting the container.

     ctr run --runtime io.containerd.runtime.v1.linux -t --env ASCEND_VISIBLE_DEVICES=0 --env ASCEND_VNPU_SPECS=vir04 image-name:tag containerID

   After the container is started, you can run the ls /dev | grep davinci* && ls /dev | grep devmm_svm && ls /dev | grep hisi_hdc && ls /usr/local/Ascend/driver && ls /usr/local/ |grep dcmi && ls /usr/local/bin command to check whether the corresponding devices and drivers are successfully mounted. For details about the mounted directories of each server model, see Default Content Mounted by the Ascend Docker Runtime.

   

   • You can query the available processor IDs as follows:
     • Physical processor ID:

       ls /dev/davinci*

     • Virtual processor ID:

       ls /dev/vdavinci*

   • image-name:tag: image name and tag, for example, ascend-tensorflow:tensorflow_TAG.
   • containerID: container ID. To start a container by running the ctr command, specify the container ID, for example, c1.
   • If you do not need to mount all content in the basic configuration file base, specify the mounted content to be reduced in hostlog. Change hostlog to the actual configuration file name.

     The mounted content of hostlog must be based on the base.list file. For details about the base.list file content, see Default Content Mounted by the Ascend Docker Runtime.

     For details about how to create a configuration file, see Creating a Configuration File. The following uses the hostlog.list configuration file as an example (The file name extension must be .list.):

   • Do not repeatedly define or fix environment variables such as ASCEND_VISIBLE_DEVICES, ASCEND_RUNTIME_OPTIONS, ASCEND_RUNTIME_MOUNTS, and ASCEND_VNPU_SPECS in the container image.

   Table 1 Parameter description

   Parameter	Description	Example
   ASCEND_VISIBLE_DEVICES	Uses the ASCEND_VISIBLE_DEVICES environment variable to specify the NPU device to be mounted to the container, and uses the device sequence number to specify the device. You can specify a single device or a device range, or both of them.	ASCEND_VISIBLE_DEVICES=0 indicates that device 0 (/dev/davinci0) is mounted to the container. ASCEND_VISIBLE_DEVICES=1,3 indicates that devices 1 and 3 are mounted to the container. ASCEND_VISIBLE_DEVICES=0-2 indicates that devices 0 to 2 (including devices 0 and 2) are mounted to the container. The effect is the same as that of -e ASCEND_VISIBLE_DEVICES=0,1,2. ASCEND_VISIBLE_DEVICES=0-2,4 indicates that devices 0 to 2 and device 4 are mounted to the container. The effect is the same as that of -e ASCEND_VISIBLE_DEVICES=0,1,2,4.
   ASCEND_RUNTIME_OPTIONS	Restricts the processor ID specified by ASCEND_VISIBLE_DEVICES. NODRV indicates that driver-related directories are not mounted. VIRTUAL indicates that the virtual processor is mounted. NODRV,VIRTUAL indicates that the virtual processor is mounted while driver-related directories are not mounted. NOTE: For details about the restrictions and creation methods of virtual devices, see "Installation in the Computing Power Allocation Scenario" in the Ascend 910 NPU Driver and Firmware Installation Guide (AI Server). For details about other product models, see the corresponding driver and firmware installation manuals.	ASCEND_RUNTIME_OPTIONS=NODRV ASCEND_RUNTIME_OPTIONS=VIRTUAL ASCEND_RUNTIME_OPTIONS=NODRV,VIRTUAL
   ASCEND_RUNTIME_MOUNTS	Reads the mounted content in the configuration file.	ASCEND_RUNTIME_MOUNTS=hostlog NOTE: Change hostlog to the actual configuration file name.
   ASCEND_VNPU_SPECS	Splits a certain number of AI Cores from a physical NPU device and specifies them as virtual devices. The value can be vir01, vir02, vir02_1c, vir04, vir04_3c, vir08, vir16, vir04_4c_dvpp, or vir04_3c_ndvpp. Ascend 310P AI Processors support only vir01, vir02, vir02_1c, vir04, vir04_3c, vir04_4c_dvpp, or vir04_3c_ndvpp. Ascend 910 AI Processors support only vir02, vir04, vir08, and vir16. This parameter must be used together with ASCEND_VISIBLE_DEVICES that specifies the physical NPU device to be split.	ASCEND_VNPU_SPECS=vir04 indicates that four AI Cores are used as virtual devices and mounted to a container. NOTE: The number following vir indicates the number of AI Cores. The number before c indicates the number of AI CPUs. dvpp indicates that all digital vision pre-processing (DVPP) modules (VPC, VDEC, JPEGD, PNGD, VENC and JPEGE) are included during virtualization. ndvpp indicates that DVPP hardware resources are excluded during virtualization.