Creating an Inference Image

Typical Service Scenarios

AI applications require the NPU computing power of edge devices. The following uses the AtlasEdge platform as an example.

Prerequisites

• Prepare the container OS image by yourself.
• Prepare the offline inference engine package and service inference program package according to the following table.

Procedure

1. Upload the software packages and scripts to the same directory (for example, /home/test).
   • Ascend-cann-nnrt_{version}_linux-aarch64.run
   • NPU driver
     

     Before building an image, copy the driver directory in the NPU driver installation path to the current directory. For the Atlas 500 AI edge station (model 3000), the NPU driver installation path is /home/data/miniD/driver. For the Atlas 500 Pro AI edge server (model 3000), use the actual NPU driver installation path.

     • To build an image on the Atlas 500 AI edge station (model 3000), run the cp -r /home/data/miniD/driver/ /home/test command.
     • To build an image on the Atlas 500 Pro AI edge server (model 3000), run the cp -r actual_NPU_installation_path/Ascend/driver/ /home/test command.
   • Service inference program package
   • Dockerfile, install.sh, run.sh
2. Perform the following steps to create a Dockerfile:
   1. Log in as the root user and run the id HwHiAiUser command to query and record the UID and GID of the HwHiAiUser user in the mirroring environment. (The UID and GID are used to replace the "gid" and "uid" in the Dockerfile compilation example.)
   2. Go to the software package upload directory in Step 1 and run the following command to create a Dockerfile:

      vi Dockerfile

   3. Enter the following content and run the :wq command to save the modification. (The Ubuntu ARM OS is used as an example, and the content is only an example. Perform secondary development as required.)

      # Change the OS and version number as required.
      FROM ubuntu:18.04
      
      # Build Arguments
      ARG NNRT_PKG
      ARG DIST_PKG
      ARG ASCEND_BASE=/usr/local/Ascend
      
      # Set environment variables.
      ENV LD_LIBRARY_PATH=\
      $LD_LIBRARY_PATH:\
      $ASCEND_BASE/driver/lib64:\
      $ASCEND_BASE/nnrt/latest/acllib/lib64
      
      ENV ASCEND_AICPU_PATH=$ASCEND_BASE/nnrt/latest
      
      # Add the HwHiAiUser user for running the service container.
      RUN umask 0022 && \
      groupadd -g gid HwHiAiUser && \
      useradd -u uid -g HwHiAiUser -m -d /home/HwHiAiUser HwHiAiUser
      
      # Set the directory for accessing the started container and the value of WORKDIR based on services. The following uses /home/HwHiAiUser/app as an example.
      RUN mkdir -p /home/HwHiAiUser/app
      WORKDIR /home/HwHiAiUser/app
      COPY $NNRT_PKG .
      COPY $DIST_PKG .
      COPY install.sh .
      
      # Install the NNRT.
      RUN bash ${NNRT_PKG} --quiet --install --install-for-all && \
      rm ${NNRT_PKG}
      
      # Install Driver
      # For the Atlas 500 AI edge station (model 3000) and Atlas 500 Pro AI edge server (model 3000), copy the driver directory in the NPU's driver installation path to the current directory in advance.
      # For the Atlas 500 AI edge station (model 3000), the NPU driver installation path is /home/data/miniD/driver. For the Atlas 500 Pro AI edge server (model 3000), use the actual NPU driver installation path.
      COPY driver/ $ASCEND_BASE/driver/
      
      #Install the dist and run the service installation script.
      RUN sh install.sh && \
          rm $DIST_PKG && \
          rm install.sh
      
      COPY run.sh /home/HwHiAiUser/app/run.sh
      RUN chmod +x /home/HwHiAiUser/app/run.sh
      
      
      RUN chown -R HwHiAiUser:HwHiAiUser /home/HwHiAiUser/app/
      RUN chown -R HwHiAiUser:HwHiAiUser $ASCEND_BASE/nnrt
      RUN chown -R HwHiAiUser:HwHiAiUser $ASCEND_BASE/driver
      USER HwHiAiUser:HwHiAiUser
      
      #Execute run.sh
      CMD bash /home/HwHiAiUser/app/run.sh

   4. After creating the Dockerfile, run the following command to change the permission on the Dockerfile:

      chmod 600 Dockerfile

   5. Modify the install.sh and run.sh scripts based on services and prepare them using the same method for preparing the Dockerfile.
   6. Compilation example of install.sh:

      #!/bin/bash
      # Access the container working directory.
      cd /home/HwHiAiUser/app
      # Decompress the service inference program package based on the package format. DIST indicates the name of the service inference program package prepared by the user.
      tar -xzvf DIST.tar

      Compilation example of run.sh:

      #!/bin/bash
      # Go to the directory where the executable file of the service inference programs is located. Use the actual path.
      cd /home/HwHiAiUser/app/FaceRecognition_b098/src/dist/HostCPU
      # Run the executable file. You need to modify the file according to the actual situation.
      ./AclFaceRecognitionHostCPU

3. Go to the directory where the software packages are stored and run the following command to create a container image:

   docker build -t image-name:tag --build-arg NNRT_PKG=nnrt-name --build-arg DIST_PKG=distpackage-name .

   If the message "Successfully built xxx" is displayed, the image is successfully built. For details about the command, see Table 3.

   

   Do not omit . at the end of the command.

   Table 3 Command parameter description

   Parameter	Description
   image-name:tag	Specifies the image name and tag. Set this parameter as required.
   NNRT_PKG	nnrt-name specifies the name of the offline inference engine package. Do not omit the file name extension. Replace it with the actual one.
   DIST_PKG	distpackage-name specifies the name of the compressed package of the service inference program. Do not omit the file name extension. Replace it with the actual one.

4. Run the following command to create a container image software package:

   docker save image-name:tag |gzip -c > image-name.tar.gz