Creating an Inference Image Using a Dockerfile

Prerequisites

The software packages of the corresponding OS and the Dockerfile and script files required for packaging images are obtained by referring to Table 1.

In the name of the offline inference engine package, {version} indicates the version, and {arch} indicates the architecture.

To prevent a software package from being maliciously tampered with during transmission or storage, download the corresponding digital signature file for integrity verification when downloading the software package.

After the software package is downloaded, verify its PGP digital signature according to the OpenPGP Signature Verification Guide. If the software package fails the verification, do not use the software package, and contact Huawei technical support.

Before a software package is used in installation or upgrade, its digital signature also needs to be verified according to OpenPGP Signature Verification Guide to ensure that the software package is not tampered with.

For carrier users, visit https://support.huawei.com/carrier/digitalSignatureAction.

For enterprise users, visit https://support.huawei.com/enterprise/en/tool/pgp-verify-TL1000000054.

Procedure

1. Upload the software packages and files to the same directory (for example, /home/infer) on the server.
   • Ascend-cann-nnrt_{version}_linux-{arch}.run
   • Dockerfile
   • install.sh
   • run.sh
   • XXX.tar (Inference code or script prepared by users)
2. Log in to the server as the root user.
3. Perform the following steps to prepare the install.sh file:
   1. Go to the directory where the software packages are stored and run the following command to create the install.sh file:

      vim install.sh

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see install.sh compilation example.
4. Perform the following steps to prepare the run.sh file:
   1. Go to the directory where the software packages are stored and run the following command to create the run.sh file:

      vim run.sh

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see run.sh compilation example.
5. Perform the following steps to create the Dockerfile file:
   1. Go to the directory where the software packages are stored and run the following command to create the Dockerfile file:

      vim Dockerfile

   2. Build a sample based on the service requirements and run the :wq command to save the content. For details, see Dockerfile compilation example.
6. Go to the directory where the software packages are stored and run the following command to create a container image. Do not omit the period (.) at the end of the command.

   docker build --build-arg NNRT_VERSION={version} --build-arg NNRT_ARCH={arch} --build-arg DIST_PKG=XXX.tar -t Image name_System architecture:Image tag .

   Example:

   docker build --build-arg NNRT_VERSION=20.1.rc3 --build-arg NNRT_ARCH=x86_64 --build-arg DIST_PKG=dvpp_resnet.tar -t ubuntu-infer:v1 .

   Table 2 describes the commands.

   Table 2 Parameters

   Parameter	Description
   --build-arg	Parameters in the Dockerfile
   {version}	Version of the offline inference engine package. Set it based on the actual situation.
   {arch}	Architecture of the offline inference engine package. Set it based on the actual situation.
   XXX.tar	Name of the inference service code package. Set it based on the actual situation.
   -t	Specifies the image name.
   Image name_System architecture:Image tag	Image name and tag. Change them based on the actual situation.

   If "Successfully built xxx" is displayed, the image has been created.

7. After the image is created, run the following command to view the image information:

   docker images

   Example:

   REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
   ubuntu-infer        v1                  fffbd83be42a        2 minutes ago       293MB

Compilation Examples

1. Compilation example of install.sh

   #!/bin/bash
   #--------------------------------------------------------------------------------
   # Install the inference service script. The following uses the inference service package dvpp_resnet.tar as an example. You can change the service package name as required.
   #-------------------------------------
   tar -xvf dvpp_resnet.tar
   # You are advised to change the permission and owner of the decompressed file.

2. Compilation example of run.sh

   #!/bin/bash
   # Run the service code.
   cd /home/out
   numbers=`ls /dev/| grep davinci | grep -v davinci_manager | wc -l`
   # Update logs every 5 minutes.
   #./main $numbers|grep -nE '.*\[.*[[:digit:]]{2}:[[:digit:]]{1}[05]:00' >./log.txt
   # Load environment variables for offline inference.
   export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/common:/usr/local/Ascend/driver/lib64/driver:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH}
   source /usr/local/Ascend/nnrt/set_env.sh
   ./main $numbers        

3. Compilation example of Dockerfile. Modify the script as required.

   # The base image ubuntu:18.04 does not contain the offline inference package Ascend-cann-nnrt-xxx.run. You can install the package by referring to this example, and you need to prepare the package in advance.
   # You are advised to pull the base inference image from the Ascend image repository. In this case, the offline inference package Ascend-cann-nnrt-xxx.run is installed in the image. Check whether the package matches the driver version on the physical machine.
   FROM ubuntu:18.04
   
   # Set the parameters of the offline inference engine package.
   ARG NNRT_VERSION
   ARG NNRT_ARCH
   ARG NNRT_PKG=Ascend-cann-nnrt_${NNRT_VERSION}_linux-${NNRT_ARCH}.run
   # Set environment variables.
   ARG ASCEND_BASE=/usr/local/Ascend
   
   # Set the directory of the started container.
   WORKDIR /home
   # Copy the driver package and offline inference engine package.
   COPY $NNRT_PKG .
   
   # Install the offline inference engine package.
   RUN umask 0022 && \
       groupadd xxx (custom user, which must be the same as that specified during driver installation.) && \
       useradd -g xxx (custom user, which must be the same as that specified during driver installation.) -s /usr/sbin/nologin (Disable user login. Ubuntu is used as an example.) -m -d /home/xxx xxx (custom user, which must be the same as that specified during driver installation.) && \
       chmod +x ${NNRT_PKG} &&\
       ./${NNRT_PKG} --quiet --install &&\
       rm ${NNRT_PKG}
   
   # Copy the service inference program package, installation script, and running script.
   ARG DIST_PKG
   COPY $DIST_PKG .
   COPY install.sh .
   COPY run.sh .
   
   # Run the installation script.
   RUN mkdir -p /usr/slog && \
       mkdir -p /var/log/npu/slog/slogd && \
       chmod u+x run.sh install.sh && \
       sh install.sh && \
       rm $DIST_PKG && \
       rm install.sh
   
   CMD bash run.sh