Creating an Inference Image Using a Dockerfile

Prerequisites

Obtain the software packages of the corresponding OS and the Dockerfile and script files required for packaging images by referring to Table 1.

In the software package name, {version} indicates the version number, {arch} indicates the architecture, and {chip_type} indicates the processor type. In CANN 6.3.RC3, 6.2.RC3, and later versions, the message "Do you accept EULA to install CANN (Y/N)" is added to the software package. In the Dockerfile compilation example, the installation command contains the --quiet parameter by default, indicating that EULA is signed by default. You can modify the parameter as required.

To avoid using software packages that have been tampered with during transmission or storage, download their digital signature files for integrity check while downloading the software packages.

After the software package is downloaded from the Support website, verify its PGP digital signature by referring to the OpenPGP Signature Verification Guide. If the verification fails, do not use the software package, and contact Huawei technical support.

The verification is also required before the installation or update of the software package.

Carriers: Visit https://support.huawei.com/carrier/digitalSignatureAction.

Enterprises: Visit https://support.huawei.com/enterprise/en/tool/pgp-verify-TL1000000054.

Procedure

1. Upload the software packages and files to the same directory (for example, /home/infer) on the server.
   • Ascend-cann-toolkit_{version}_linux-{arch}.run
   • Ascend-cann-{chip_type}-ops_{version}_linux-{arch}.run
   • Dockerfile
   • install.sh
   • run.sh
   • XXX.tar (Inference code or script prepared by yourself)
2. Log in to the server as the root user.
3. Perform the following steps to prepare the install.sh file:
   1. Go to the directory where the software packages are stored and run the following command to create the install.sh file:

      vi install.sh

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see install.sh compilation example.
4. Perform the following steps to prepare the run.sh file:
   1. Go to the directory where the software packages are stored and run the following command to create a run.sh file:

      vi run.sh

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see run.sh compilation example.
5. Perform the following steps to create a Dockerfile:
   1. Go to the directory where the software packages are stored and run the following command to create a Dockerfile:

      vi Dockerfile

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see Dockerfile compilation example.
6. Go to the directory where the software packages are stored and run the following command to create a container image. Do not omit the period (.) at the end of the command.

   docker build --build-arg TOOLKIT_VERSION={version} --build-arg TOOLKIT_ARCH={arch} --build-arg DIST_PKG=XXX.tar -t Image name_System architecture:Image tag .

   The following table describes the command parameters.

   Table 2 Parameters in the commands

   Parameter	Description
   --build-arg	Parameters in the Dockerfile
   {version}	ToolKit package version. Set it based on the actual situation.
   {arch}	ToolKit package architecture. Set it based on the actual situation.
   XXX.tar	Name of the inference service code package. Set it based on the actual situation.
   -t	Image name.
   Image name_System architecture:Image tag	Image name and tag. Change them based on the actual situation.

   Example:

   docker build --build-arg TOOLKIT_VERSION=20.1.rc3 --build-arg TOOLKIT_ARCH=x86_64 --build-arg DIST_PKG=dvpp_resnet.tar -t ubuntu-infer:v1 .

   If Successfully built xxx is displayed, the image has been created.

7. After the image is created, run the following command to view the image information:

   docker images

   Command output:

   REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
   ubuntu-infer        v1                  fffbd83be42a        2 minutes ago       293MB

Compilation Examples

1. Compilation example of install.sh

   #!/bin/bash
   #--------------------------------------------------------------------------------
   # Install the inference service script. The following uses the inference service package dvpp_resnet.tar as an example. You can change the service package name as required.
   #-------------------------------------
   tar -xvf dvpp_resnet.tar
   # You are advised to change the permission and owner of the decompressed file.

2. Compilation example of run.sh

   #!/bin/bash
   # Run the service code.
   cd /home/out
   numbers=`ls /dev/| grep davinci | grep -v davinci_manager | wc -l`
   # Update logs every 5 minutes.
   #./main $numbers|grep -nE '.*\[.*[[:digit:]]{2}:[[:digit:]]{1}[05]:00\]' >./log.txt
   # Load environment variables for offline inference.
   export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/common:/usr/local/Ascend/driver/lib64/driver:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH}
   source /usr/local/Ascend/cann/set_env.sh
   ./main $numbers        

   

   Driver-related paths are configured in LD_LIBRARY_PATH. Files in the paths are used when an inference job is executed. It is recommended that the running user of the inference job be the same as that specified during driver installation to avoid privilege escalation risks caused by user inconsistency.

3. Compilation example of Dockerfile. Modify the script as required.

   # The basic image ubuntu:18.04 does not contain the ToolKit package. You can install it by referring to some steps in the Dockerfile example. The ToolKit package must be prepared in advance.
   # It is recommended that you pull the inference base image from the Ascend image repository. The ToolKit package has been embedded in the image. In addition, check whether the ToolKit package matches the driver version on the physical machine.
   FROM ubuntu:18.04
   
   # Set the parameters of the ToolKit and OPS packages.
   ARG TOOLKIT_VERSION
   ARG TOOLKIT_ARCH
   ARG TOOLKIT_PKG=Ascend-cann-toolkit_{version}_linux-{arch}.run
   ARG OPS_PKG=Ascend-cann-{chip_type}-ops_{version}_linux-{arch}.run
   
   
   # Set environment variables.
   ARG ASCEND_BASE=/usr/local/Ascend
   
   # Set the directory of the started container.
   WORKDIR /home
   
   # Copy the ToolKit and OPS packages.
   COPY $TOOLKIT_PKG .
   COPY $OPS_PKG .
   
   # Install the ToolKit and OPS packages.
   RUN umask 0022 && \
       groupadd xxx (custom user, which must be the same as that specified during driver installation.) && \
       useradd -g xxx (custom user, which must be the same as that specified during driver installation.) -s /usr/sbin/nologin (Disable user login. Ubuntu is used as an example.) -m -d /home/xxx xxx (custom user, which must be the same as that specified during driver installation.) && \
       chmod +x ${TOOLKIT_PKG} &&\
       ./${TOOLKIT_PKG} --quiet --install --install-for-all --whitelist=nnrt --force &&\
       rm ${TOOLKIT_PKG}
       chmod +x ${OPS_PKG} &&\
       ./${OPS_PKG} --install --install-for-all --quiet --force &&\
       rm ${OPS_PKG}
   
   # Copy the service inference program package, installation script, and running script.
   ARG DIST_PKG
   COPY $DIST_PKG .
   COPY install.sh .
   COPY run.sh .
   
   # Run the installation script.
   RUN mkdir -p /usr/slog && \
       mkdir -p /var/log/npu/slog/slogd && \
       chmod u+x run.sh install.sh && \
       sh install.sh && \
       rm $DIST_PKG && \
       rm install.sh
   
   CMD bash run.sh

   

   If the CANN package version is 6.2.RC1, 6.3.RC1, or later, add the --force parameter when installing the package. This parameter has been added to the preceding Dockerfile compilation example. If you use the software package of a version earlier than 6.2.RC1 or 6.3.RC1, delete this parameter from the Dockerfile compilation example.